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Abstract 



This report develops an analytical framework based on systems analysis for evaluating 
electronic question/answer or AskA services operated by a wide range of types of organizations, 
including libraries. Version 1.0 of this framework was applied in June 1999 to a selective 
sample of eleven electronic question/ahswer services, which cover a range of primary audiences, 
intended purposes, and sponsoring organizations. All are based in non-library organizations, 
except for the help desk of the American Memory Project at the Library of Congress. Data 
gathering consists primarily of web site inspection, perusal of policy documents if publicly 
available, and personal contact via e-mail and/or interviews with service administrators. 

Contents include: Analyzing Electronic Question/ Answer Services: Introduction and 
Analytical Framework by Marilyn Domas White, and the following evaluative analyses: 
American Memory Historical Collections Help Desk, American Memory Project, Library of 
Congress by Eileen Hauser; Ask-A-Geologist (U.S. Geological Field Service, Menlo Park, CA) 
by Janet Spikes; Ask-a-Linguist by Margaret F. Kondash; Ask a NASA Scientist by Theresa L. 
Aquino; Ask Dr. Math by Laura W. Speer; Ask Dr. Universe by Kristen Gramer; Go Ask Alice! 
by Vera Welsh; KidsConnect by Kathleen Feeney Chappell; LawGuru.com BBS by Bryan 
Fagan; The Medical Edge’s AnswerDoc by Rick Whitaker; and ScienceNet by Jennifer R. 
Heiland. Appendix A contains the original framework; the introductory chapter presents 
Version 1.1 of the evaluative framework as modified through these applications. 

Keywords: Digital libraries; Digital reference service; AskA services; Electronic 
question and answer services; Evaluations; Evaluative criteria 



Analyzing Electronic Question/Answer Services: 
Introduction and Analytical Framework 

by Marilyn Domas White 



Introduction and Literature Review 

To be responsive to the needs of 
information seekers operating within digital 
libraries, reference services are evolving 
and taking on different forms. They are 
based in a wide range of organizations, 
including libraries. They are sometimes 
oriented to matching people with questions 
to people with expertise to answer those 
questions, not just to matching an 
information need to a textual source with 
the information. They are responding with 
information presented through a range of 
visual and aural media. They are staffed by 
people with a broad range of education and 
experience, including, but not limited to, 
professional librarians. They give the client 
considerable time and location flexibility in 
submitting questions; people can ask 
questions when they experience them and 
contact the service they consider optimal. 
And, unfortunately, they are dependent to a 
large extent on receiving queries through 
information channels that do not provide the 
rich visual and aural clues that abound in 
face-to-face encounters and do not easily 
accommodate conversations that clarify the 
information need. The electronic 
question/answer services discussed in this 
paper are good examples of the new 
services that are being developed. 

The purpose of this report is to present a 
framework for analyzing and evaluating 
electronic question/answer services and to 
use it to analyze a wide range of electronic 
question/answer services. 

Most of the literature related to 
individual electronic question/answer 
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services is descriptive, often written by 
service administrators, and emphasizes 
procedures and/or policies. See, for 
example, the case studies included in David 
Lankes and Abby Kasowitz's AskA Starter 
Kit: How to Build and Maintain Digital 
Reference Services (1998) (Bennett 1998; 
Bry 1998; Lynch 1998; Odenwald 1998; 
Stahl 1998a; Weissman 1998; Williams & 
Weimar 1998), also Stahl, 1998b. These 
analyses are useful but do not allow for easy 
comparisons across services because they 
do not always contain common elements. 

David Lankes’ dissertation. Building 
and Maintaining Internet Information 
Services (1998), on the other hand, applies a 
common framework to analyze 
systematically six AskA services serving 
primarily the K-12 community: Ask a 
Volcanologist, Ask Dr. Math, Ask Shamu, 
How Things Work, Mad Scientist Network, 
National Museum of American Art 
Reference Desk. Drawing on Holland's 
performance system of agents (1995) and 
general systems theory (Bertalanffy 1968), 
Lankes' framework elicits detectors, rules, 
and effectors of an agent’s performance 
system. The agent in this context is the 
electronic question/answer service. 
Detectors are the “selective sensors used to 
gather information (Lankes, p. 46),’’ and 
effectors are “the services, or interfaces, 
offered to users and other agents (Lankes, p. 
48).’’ 

Other literature is broader and focuses 
on critical factors affecting digital reference 
services (See, for example, the papers 
presented at the Library of Congress 
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institute, Reference Service in a Digital Age 
1998). 

Evaluative Analyses 

The individual chapters in this report 
are evaluative analyses of electronic 
question/answer services prepared by 
students in LBSC 758, Seminar in 
Information Access, a course offered in the 
College of Library and Information 
Services, University of Maryland, in June 
1999. 

Services selected 

The eleven services examined in the 
chapters constitute a selective sample of 
existing services and were purposefully 
chosen to represent active, ongoing services 
with some history of service and, if 
possible, with adequate documentation at 
their web sites, including policy and/or 
procedures documents and archives or 
FAQs that would indicate the nature of the 
questions and responses. Not all of the 
services met all of these criteria. All are 
based in non-library organizations, except 
for the help desk of the American Memory 
Collections Project at the Library of 
Congress. Although several papers examine 
widely known and reported AskA services 
such as Ask Dr. Math and KidsConnect, 
others evaluate services that have received 
little or no previous attention in the 
literature. Three services have adults as 
their primary clientele: Ask-a-Linguist, 
LawGuru.com BBS, and The Medical 
Edge’s AnswerDoc. Two services, 
LawGuru.com BBS and AnswerDoc, are 
clearly intended as public service initiatives 
for commercial organizations. 

Data gathering 

The analyses are based on data 
gathering that occurred between June 3 and 
June 9, 1999. Data gathering techniques 
included analysis of the web site, first-hand 
use of the services, perusal of the archives 
and/or FAQs, analyses of policy and or 
administrative documents, and, in many 



cases, contacts with service administrators. 
Not all techniques could be used or were 
used for each service because sometimes 
elements, such as archives or policy 
documents, were not publicly available. 

To guide data gathering, all of the 
authors followed the Framework for 
Analysis of an Electronic Question/Answer 
Service, Version 1.0 (June 4, 1999) (See 
Appendix A). The latter part of this chapter 
presents the revised Version 1.1 of this 
framework, which includes modifications 
based on insights gained from the analyses 
presented in this report. 

Presentation formats 

No guidelines were established for 
presenting analyses of the different 
electronic services, so the formats of the 
individual chapters vary — some present the 
information in a narrative form; others 
follow a question/answer format, with the 
questions representing points in the 
framework. In some cases authors’ 
evaluative comments are separate and/or 
indicated by bolding or by the word 
“Comments” preceding the section. Yet 
others interweave critical comments 
throughout the paper. 

Importance of the evaluations and the 
framework 

The value of these evaluations accrues 
from several factors: 

• They follow a common and 
comprehensive analytical framework, 
which allows for indicating not only 
what is available for the service, but 
also, and perhaps more importantly, 
what is not available. 

• They are both descriptive and critical. 

• They encompass both well-known and 
less well-known or newer services. 

• They cover services provided by non- 
profit organizations as well as services 
that are public service initiatives for 
commercial organizations. 
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Using an analytical framework results 
in several significant benefits. First, it 
establishes the basis for noting strengths and 
weaknesses in existing services. Second, it 
reveals gaps in coverage or implementation 
of key elements in electronic 
question/answer services. Third, although it 
was not done systematically in this report, a 
common analytical framework supports 
comparisons across individual services. A 
useful outcome of such comparisons is the 
delineation of models of electronic 
question/answer services, i.e. standard 
variations of organizations or operations for 
these services. Another benefit is the 
identification of “best practices" for 
electronic question/answer services. 

Framework for Analysis 

The original version of the framework 
was based on systems analysis with the 
emphasis on identifying key elements 
associated with client input, the thruput, i.e. 
the processing of the query by the system, 
and system output, i.e., communications 
from the service, usually in the form of 
responses to queries. In applying the 
framework, the authors noted the benefits of 
using it as a guide not only for looking 
critically at existing systems but also for 
developing new services. 

A modified version, shown in Figure 1, 
is based on a systems analysis of a 
generalized electronic question/answer 
service plus the systematic analyses of the 
eleven services covered in this report. 
Questions were added and/or modified to 
allow for the variants that were noted in 
these services. In addition, the most logical 
category for each question was determined 
and redundancies in the original framework 
were eliminated. Finally, the order of 
categories was modified to provide a more 
coherent approach to the analysis. 

The framework includes about 150 
questions related to 18 basic categories: 



Mission, Objectives, Statement of 
Purpose 

Parameters of Service: Questions 
Parameters of Service: Clients 

Administration 

Staffing and Training 

Hardware and Software 

Ease of Use, Instructions to the Client 

Responsibilities to the Client 

Query Form 
Acknowledgment 
Question Negotiation 
Question Answering Process 
Response Guidelines 
Coping with Demand 
Archiving 

Quality Control 
Evaluation 
External Recognition 

As grouped above, the first set identifies 
the purpose of the service, broadly and 
specifically; the second group identifies the 
basic administrative structure and the 
system's responsibilities to the client; the 
third focuses directly on submitting, 
receiving, and responding to questions; and 
the fourth group returns to a broader 
perspective, looking at monitoring the 
quality and effectiveness of the service. It 
should be noted that this order is not the 
order in the original version and thus not the 
order that is generally apparent in the 
individual chapters. 

The questions in the framework usually 
elicit factual information, not judgments. 
This information in turn establishes the 
basis for making judgments about the merits 
of the service as a whole and of the sub- 
parts of the service. The information 
reflects management decisions in most 
cases, and these should be subject to careful 
scrutiny. In the evaluations included in this 
report, the authors not only provide 
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information but also at times express their 

judgments about the various features of the 

services. 

Figure 1. Framework for Analysis of an 
Electronic Question/Answer Service 
Version 1.1 (July 25, 1999) 

1. Mission, Objectives, Statement of 
Purpose 

1 . 1 What is the name of this service? 

1 .2 Is the name expressive? 

1.3 What is the URL? 

1 .4 What is the mission/purpose of this 
service? 

1 .5 Is the mission/purpose apparent to 
the client? 

2. Parameters of Service: Questions 

2.1 What types of questions does this 
service answer? For example, short- 
answer questions, factual questions, 
questions related to a specific 
subject, unique questions, i.e. those 
not already in the archive. 

2.2 Does the service reject questions? If 
so, what is the basis for rejection? 
For example, outside of scope, time 
limited. 

2.3 What limits exist on the service for 
an individual client? Consider, for 
example: 

- number of questions per message 

- number of questions per client 

2.4 If questions are not appropriate for 
the service, are the clients referred 
elsewhere? 

3. Parameters of Service: Clients 

3.1 Who are the intended clients? What 
are their characteristics? Consider, 
for example, age, gender, affiliation, 
location. 

3.2 Are the intended clients clearly 
identified at the site? 

3.3 If appropriate, is the site clearly 
oriented to that client group? For 
example, illustrated accordingly for 



children, written at an appropriate 
level. 

3.4 Are any categories of clients 
dissuaded or barred from using the 
service? For example, scholars in 
the field for a service oriented to lay 
persons. 

3.5 If clients are not eligible for the 
service, are they referred 
elsewhere? 

4. Administration 

4.1 How is the service funded? 

4.2 How does funding/sponsorship 

. influence quality and/or objectivity? 

4.3 Is service fee-based? If so, what is 
the fee structure and how are the 
fees collected? 

4.4 Are there ethical guidelines? For 
example, guidelines about questions 
requesting dangerous information. 
Are they publicly accessible? If so, 
how easy are they to locate? 

4.5 Are there privacy guidelines? Are 
they publicly accessible? If so, how 
easy are they to locate? 

4.6 How are copyright and fair-use 
issues managed? 

4.7 Are there liability disclaimers 
and/or content disclaimers? 

4.8 What other policy and/or procedure 
documents are available online? 

Are they publicly accessible? If so, 
how easy are they to locate? 

4.9 If access is restricted for certain 
policy and procedural documents, 
what documents exist and who can 
access them? For example, some 
types of documents may only be 
available to volunteers. 

4.10 How is policy decided? 

4.11 To what extent are staff involved in 
policy development? 

5. Staffing and Training 

5.1 How is the staff organized? 

5.2 Who answers the questions? For 
example, volunteers, in-house staff 

5.3 What level of expertise is required? 
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5.4 How are answerers recruited and 
selected? 

5.5 What is expected/average time 
commitment for answerers? 

5.6 What training exists for the 
answerers? 

5.7 Are there seniority benefits and/or 
responsibilities? 

5.8 If answerers are staff, how do the 
Q/A service responsibilities fit into 
their other responsibilities? 

6. Hardware and Software 

6. 1 Describe the hardware used by this 
system. 

6.2 Describe the software used by this 
system. 

7. Ease of Use, Instructions to Client 

7. 1 How is the service positioned on the 
web page in relation to the other 
services on the site? 

7.2 Is it easy or difficult to find? 
Consider logic of links, name of 
service. 

7.3 Does the web site direct the client to 
other relevant sources? For 
example, FAQs, archives, links to 
other services. 

7.4 Is the destination address apparent 
to the client of a query form? 

8. Responsibility to Clients 

8.1 What does the service indicate it 
will provide to the client? 

8.2 Does it inform the client about: 

- how soon an answer is returned? 

- who answers the question? For 
example, volunteers, professionals, 
experts, other clients. 

- the types/or specific sources it 
uses? 

- the percentage of questions 
actually answered? 

For any of these, where and how? 

8.3 Does it explain the question- 
answering process? Where and 
how? 




8.4 Does the service address client 
privacy? How? 

8.5 Is there a statement of liability/ 
responsibility/appropriate use (for 
example, medical, legal 
information, accuracy)? 

8.6 If so, what is included in it? 

9. Query Form 

9.1 What information is required to 
answer a question/What must the 
client provide? For example, name, 
e-mail address, phone number, 
question, sources already used, date 
needed by, grade 

9.2 What options are available for 
submitting a question? For 
example, e-mail, query form, both. 

9.3 What guidance is given on how to 
ask a question? For example, 
examples of questions, complete 
sentences only. 

10. Acknowledgment 

10.1 What kind of acknowledgment is 
sent when a query is received, if 
any? 

10.2 How soon is it sent? 

10.3 What information does it contain? 
For example, 

- restatement of the question 

- expected response time 

10.4 Is the acknowledgment generated 
automatically or with human input? 

11. Question Negotiation 

11.1 If the question requires clarification, 
what are the guidelines for 
clarification? Consider: 

- the number of messages needed to 
clarify the question 

- modes to be used for 
communicating 

- any limitations on time 

12. Question-Answering Process 

12. 1 Describe the process a question 
goes through from receipt to 
response. 
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12.2 Do questions accumulate? Where? 
For how long? 

12.3 How are questions prioritized? 

What criteria are used? 

12.4 Are questions assigned or selected 
by answerers? 

12.5 If assigned, who does the assigning? 
12.6. On what basis are questions 

assigned and/or selected by the 
answerers? 

12.7 What is done with questions or 
clients not matching the service 
parameters? 

12.8 Is the client apprised of actions at 
critical stages in the answering 
process? For example, when a 
question is referred to another 
answerer after an unsuccessful 
search. 

1 2.9 If not, how important are these 
considering 

- length of normal response time 

- percentage of questions answered 

12.10 What are the procedures if an 
answerer cannot answer the 
question? Note: May be because he 
cannot find information, does not 
have time, etc. 

12.11 Who monitors the process to ensure 
that the system is following 
procedures and meeting 
expectations? 

13. Response Guidelines 

13.1 How is the response generated: 
form, free writing, or combination 
of both? 

13.2 Does the service use standard 
language to respond in full or part 
to questions? If so, for what kinds 
of questions? If so, solicit 
examples. 

13.3 What are the limitations or con- 
straints on the answers? Consider 
issues of 

- brevity 

- fact/judgment 

- for bibliographic citations: 
number and format 



- instruction, actual information or 
both. If the latter, what factors 
influence the judgment about which 
to provide? 

- level of difficulty 

degree of detail 

13.4 Is the original question included 
with the response? How? For 
example, repetition of the client’s 
language or re-statement to indicate 
the answerer’s understanding of the 
query. 

13.5 What documentation accompanies 
the answer, if any? 

13.6 How does the service take into 
consideration factors that may 
influence choice of answer? 
Consider as factors, for example, 
level of knowledge or language 
skills of client. 

13.7 What policy restrictions are there on 
the sources and contact that can be 
used to answer the questions, if 
any? 

13.8 Are previous answers available and 
searchable so that answers can be 
reused or adapted for other ques- 
tions? 

13.9 What format does the response take: 
direct e-mail or bulletin board 
posting? 

13.10 Is there a policy on response time 
for question answers? What is 
considered the range of acceptable 
response times? 

13.11 What are the policies regarding 
attaching or referring to non-textual 
materials? 

13.12 How much personalization is 
allowed in responses? 

13.13 Are personal contacts between 
answerers and questioners allowed 
or encouraged? 

13.14 What is the policy and procedure 
for referring a question, if any? To 
whom are referrals made? 

13.15 What are the guidelines on the 
amount of time spent on answering 
an individual question, if any? 
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13.16 How does the client request 
clarification of the answer? 

13.17 How does the client request follow- 
up? 

13.18 Who responds to requests for 
clarification or other follow-ups? 

13.19 Is there a policy on response time 
for clarifications or other follow- 
ups? What is considered the range 
of acceptable response times? 

14. Coping with Demand 

14. 1 Are there scheduled down-times? 

14.2 How are down-times and other 
interruptions of service com- 
municated to clients? 

14.3 Is there a limit to the number of 
questions the system can handle? 
What is that limit? 

14.4 What is done if system is inundated 
by demand? Are there backup staff, 
is it closed down, is there a delayed 
response time, are questions not 
answered, etc.? 

15. Archiving 

15.1 What is the nature of the archive of 
questions and answers, if there is 
one? Consider: 

- private/publicly accessible. If 
private, accessible to whom? 

- kinds of information included 

- degree of anonymity maintained 

- selectivity. If selective, on what 
basis? 

- length of time materials are 
archived 

- purpose of archive 

15.2 Are clients allowed to specify if 
they do not want their question 
archived? 

1 5.3 How easily accessible is the archive 
if it is public? 

15.4 Is archived indexed and/or 
searchable? If so, how? 

15.5 Is archive selective? By date, or 
content? 

1 5.6 Is there a FAQ? Is it indexed or 
searchable? 



15.7 How is FAQ related to archive? 

1 5.8 Who manages the archive? 

1 6. Quality Control 

1 6. 1 How is quality control maintained? 
Consider, for example, using 
standard resources for answering 
questions (written, people, 
electronic). 

16.2 What happens if quality is 
considered inadequate? 

16.3 Is there a review process for new 
answerers, for answers? If so, 
describe it, including: 

- who does it 

- what standards are applied 

- for how long is this done. 

- who or what determines when it is 
no longer necessary 

17. Evaluation 

17.1 Is there a formal evaluation 
procedure in place? 

17.2 What is the nature of that 
procedure? 

17.3 How frequently does it occur? 

17.4 What measures of performance are 
used? Consider, for example: 

- client satisfaction 

- accuracy 

- time lag 

- response rate (percentage of 
questions answered) 

17.5 Is there a regular follow-up with the 
clients? If so, in every case or in 
selected cases? 

If selected, on what basis are cases 
selected? 

17.6 How does the service determine if it 
meets mission goals? 

17.7 What statistics are maintained by 
the service regularly? How are they 
used? 

18. External recognition 

18.1 What awards has this site received? 

18.2 What articles are written about it? 

If more than five, limit by authority 
and/or date. 
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American Memory Historical Collections Help Desk, 
American Memory Project, Library of Congress: 
Evaluative Analysis 

by Eileen Hauser 



Mission, Objectives, Statement of 
Purpose 

The American Memory Collection is a 
part of the National Digital Library Program 
(NDLP) at the Library of Congress (LOC). 
The multimedia collection includes 
digitized documents, photographs, sound 
recordings and moving pictures reflecting 
the attitudes, perspectives, and beliefs of 
America from its infancy that would not 
otherwise be accessible to the public. The 
exhibitions include digital collections from 
other institutions as well that complement 
LOC’s own resources. 

Since this collection and service are 
sponsored by the Library of Congress, it is 
useful to examine the vision of that 
organization since its foresight set the stage 
for the NDLP. The Library of Congress is 
“to make its resources available and useful 
to Congress and the American people and to 
sustain and preserve a universal collection 
of knowledge and creativity for future 
generations (http://lcweb.loc.gov/ 
ndl/mission.html).” Furthermore, as stated 
in the published priorities, “the unifying 
purpose of providing the public with 
essential library services, such as cataloging 
and reference help, is to afford as much 
access to useful information as possible ... 
(http://lcweb.loc.gov/ndl/mission.html).” To 
support these priorities, LOC will maintain 
an infrastructure that will provide and 
deliver electronic services in order to 
execute “the Library’s mission and 
priorities with speed, quality, and economy 
(http//lcweb.loc.gov/ndl/mission.html).” 



Aligned with this mission, the National 
Digital Library Program offers “broad 
public access to a wide range of historical 
and cultural documents as a contribution to 
education and lifelong learning (http: 
//memory.loc.gov/ammem/dli2/html/lcndlp. 
html).” 

The primary historical documents 
accessible through the American Memory 
Collection include over fifty online 
collections, for example, “Selected Civil 
War Photographs, 1861-1865,” “Baseball 
from the 1860’s to the 1960’s,” and 
recording of historical speeches. 
Competitors from outside agencies all over 
the country submit their proposals to the 
NDLP. Annual winners are selected to 
participate in this collaborative effort. As a 
result, some of the collections are housed at 
LOC; others are physically at other host 
institutions such as a university library, but 
they are accessible through this unique 
gateway. 

The mission statement of the National 
Digital Library e-mail reference service has 
recently been developed (although it is not 
posted as of June 1999). Its first aim is: 

to provide accurate, timely, on-line 
assistance for patrons utilizing the 
American Memory Historical 
Collections, the Learning Page, and 
Today in History. 

The service also responds to inquiries 
about the National Digital Library Program. 
Secondary in the mission is to assist users 
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in finding the information they need about 
the Library of Congress collections, 
programs and services. The third aim, as 
resources allow, is to assist patrons in 
identifying information resources outside 
the National Digital Library and the 
American Memory Historical Collections 
that assist them in finding the information 
they need (Interview with service 
administrator, June 1999). 

Ease of Use 

Accordingly, the name of the digital 
reference service attached to this collection 
is the American Memory Historical 
Collections Help Desk, and this service 
deals specifically with questions about the 
American Memory collections. The 
availability of this service is not advertised 
nor displayed prominently on the front 
pages of the American Memory Collection 
web pages; in fact, it is a bit hidden. There 
are several steps that a user must take to 
gain access to the e-mail reference service: 

1 . A user first links to the URL for 
American Memory 

( http//www. loc . gov/am m em . 

2. To gain access to the e-mail reference 
service, a first time user must link to 
American Memory Help Desk at 
(http//memory. loc.gov/ammem/helpdes 
k); the link is at the bottom of the page. 

3. The user is then linked to the FAQ page 
(http//memory.loc.gov/ammem/helpdes 
k/amfaq.html). 

4. The final question in the FAQ list will 
prompt a link to the e-mail reference 
form (http/Zmemory.loc.gov/ 
ammem/helpdesk/amform.html) 

The procedure of taking the patron 
through these sites was instituted about 
eight months ago to help the user 
understand the limitations of the service 
before asking a question. It also eliminated 
much of the site’s “spam” because the e- 
mail address itself is not listed as the link. 
The words American Memory Help Desk 



serve as the words on this link (except from 
the Learning Page.) Unfortunately, from 
the users’ point of view, these numerous 
steps all but discourage the user from 
realizing that they have access to an on-line 
reference service. 

Parameters of Service 

Currently, there is no written policy 
defining parameters of this e-mail reference 
service. Instead there are unwritten 
“generally agreed upon criteria (Interview 
with service administrator, June 1999)” to 
guide the librarians in staffing this service. 
These mental guidelines are similar to the 
parameters posted under “Guidelines for 
Researchers” which describe the scope of 
the LOC reference service in 
general(http//lcweb.loc.gov/rr/refcorr.html) 

The policy of the LOC is to encourage 
correspondents to use local libraries first, 
such as public, state, academic and special 
libraries which can respond more quickly 
and thoroughly and are better able to 
identify and respond to specific research 
needs. The scope of service does not 
include: 1) compilation of bibliographies; 
2) responses to request for information 
connected with school assignments, or 
contests; and 3) research in heraldry or 
family history. Due to the unique nature of 
NDPL collections, however, this service is 
often the only resource available to the 
general public to pursue information on 
such specialized topics, a fact that is taken 
into consideration in deciding if the 
question will be answered. 

Query Form 

The form requests just enough 
information to guide both the patron in 
asking and the librarian in answering the 
query. The question or comment form 
requests the following information from the 
user (the only required entry is the e-mail 
address): 
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1. Name. 

2. E-mail address. 

3. Type of question/comment (choices of 
reference question, general comments, 
error or problem). 

4. Subject of the question or comments. 

5. Actual question or comment in a 2 x 5 
inches area with a prompt requesting 
keywords or phrases that describe the 
topic. 

6. Reason for researching topic. This area 
includes the choices of general 
interest/lifelong learning, school 
project, academic research, classroom 
instruction along with the opportunity to 
choose a description educational level 
of question. 

7. User can also indicate which American 
Memory Historical Collections have 
been searched with choices provided for 
easy click on choices. 

8. Send/Clear forms. 

Responsibility to Clients 

When the patron can expect an answer 
is delineated below the form: “We will 
make every attempt to respond to your 
question within 5 working days.” 
Nevertheless, there is no indication to the 
patron of the type of answer he will receive 
to his query. Adding such a feature would 
enlighten the user as to what he can expect 
in the reply. 

At this time, there is no automated 
response system in place for the American 
Memory Collection. In contrast, LOC does 
have an auto-response system in place, and 
the NDLP would like to see a similar 
system in theirxollections. Clients are able 
to submit questions at any time, however 
the response time is limited to business 
hours. 

Questions from patrons posted through 
this service are never ignored. If the 
question can be answered elsewhere or is 
beyond the scope of the service, the patron 
will receive a standard rejection message. 



As an example, a question such as “Who 
was the commanding general in the Battle 
of the Bulge?” would receive the following 
response 

Thank you for visiting the 
American Memory Historical 
Collections. The question you have 
asked is outside of the scope of this 
reference service and not contained 
in the American Memory Historical 
Collections. A librarian in your 
school or public library can assist 
you in identifying resources that 
will help you find the information 
you requested. 

Ideally, an automatic acknowledgement 
system would be put in place to send to 
patrons whose questions will be answered 
while sending the rejection message to 
others so that they could pursue other 
sources of information. 

When necessary, the librarians will 
negotiate a reference interview via e-mail, 
although the librarians report that only 50 
percent respond to this attempt at 
clarification. The answers provided can 
refer to other web links, virtual reference 
desks, and known expert sites; however, 
they will not intentionally refer an online 
patron to another AskA service. 

Staffing and Training 

The librarians that answer the questions 
must provide this service in addition to their 
other duties. Two librarians respond to e- 
mail questions, and they estimate that this 
service takes up about 10 percent of their 
day since volume is not high at this time. 
Nevertheless, the progress on research and 
development of this project is hampered 
because of time and personnel restraint. 
Routinely, the e-mail queries are reviewed 
in the morning, and limited time is spent on 
individual questions. At this time, there is 
no special training, and the librarians simply 
use their reference skills online. 
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Answering the Questions 

After the web form is sent by the user, it 
goes to a perl script program written by a 
National Digital Library staff member 
which sends it into their reference e-mail 
inbox where one of the two librarians reads 
it, responds, refers or forwards as 
appropriate. An internal e-mail system 
provides easy access to collection experts so 
that the librarian can get an answer for the 
patron if the question is beyond their own 
expertise. With the off-site collaborative 
collections, the librarian can contact the 
“expert” at that location and utilize their 
knowledge, if necessary. 

Usually, priority for library services at 
LOG is given to members of Congress and 
the U.S. Government; however, since this is 
not the mode which these patrons would use 
to get information, there is no priority 
consideration for questions received by the 
reference service, other than the order in 
which they were received. 

The desired format is for the question to 
be re-stated in the response; however, there 
is no formal or standardized procedure in 
place regarding the nature of the response 
for the service at this time. 

The librarians at this service do 
maintain a degree of formality when 
addressing users. Usually, determination is 
made as to the gender based on the name 
submitted and a response is composed 
beginning with “Dear Mr. or Mrs.” 
accordingly. The librarians do not sign their 
name to the response nor do they provide 
any further contact information. 

Liability/Responsibility/Appropriate Use 

Information on this area is found in 
several places but does not specifically 
relate to that of the digital reference service. 
On several pages there is a red box 
containing the word “Notice” linking to a 
statements relating to this area. The library 
advises users on copyright and restrictions 



to guide the appropriate use of an item but 
leaves the responsibility for appropriate use 
to the user. See http//memory. loc.gov/ 
ammem/copyrit2.html. Additionally, in the 
pages of the various collections, a 
disclaimer is provided to the user: 

The Library of Congress presents 
these documents as part of the 
record of the past. These primary 
historical documents reflect the 
attitudes, perspectives, and beliefs 
of different times. The Library of 
Congress does not endorse the 
views expressed in these 
collections, which may contain 
materials offensive to some readers. 

Funding 

The service is funded as part of the 
National Digital Library Project sponsored 
by several corporations in conjunction with 
the Library of Congress. A list of sponsors 
is available at http/Zmemory/loc.gov/ 
ammem/sponsors.html. Currently, the 
Development Office is looking for outside 
assistance to sponsor the e-mail reference 
aspect of the project since growth and 
development is envisioned over the next 
year. 

Statistics 

A monthly report using the questions 
and answers in a computer folder provides 
the activity report. At this time, no statistics 
are kept about users. At some point, the 
service may use the information provided 
by the user to profile user characteristics. 

Software 

The software currently used for this e- 
mail reference service is Eudora Light, 
however, this will be switched to Microsoft 
Outlook in the near future since the project 
just received funds from Microsoft. 

Archiving 

Plans for developing a searchable 
question archive are in the works. Such a 
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followed up by personal attention (Interview 
with service administrator, June 1999).” 



database would be tremendously helpful to 
both staff and patrons as it would serve as 
an additional resource. Personal 
information from archived questions would Sources 

be removed. Interviews with Linda White, Digital 

Library Specialist, Library of Congress, 
Quality Control June 1999. 

At this time, there are no controls in 
place, and there is no review system for the 
digital reference service itself. Comments 
from patrons are recorded and plans to 
explore devices to measure customer 
satisfaction aspects of the service are also 
planned for the future. 

An error and comment form comes into 
the same address also. The American 
Memory Quality Team was developed to 
correct errors reported through this method. 

Error reports are forwarded to these team 
members, which evaluates and corrects 
them if necessary. Compliments from users 
are also received via this form 
(http//m emory . loc .gov/am mem/ 
helpdesk/error.html). 

Future Developments 

Because this service is still under 
development, many of the gold standard 
aspects of a digital reference service that 
might be expected from a national library 
are not yet in place. David Lankes and 
other AskA experts met in Spring 1999 to 
guide and plan the digital reference service 
for this collection. At one time, a 24-hour- 
a-day, 7-day-a-week service was under 
consideration but will apparently not come 
to fruition. 

As policies are written and guidelines 
set, the next year will bring a more 
developed service including an archive. 

The planners are aware of what is necessary 
for an effective digital reference service, 
however, fitting research and development 
within their daily tasks slows down 
planning. Ultimately, they are “looking 
forward to a query driven knowledge 
database search for instantaneous results 
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Ask-A-Geologist (U.S. Geological Survey, Menlo Park, CA): 

Evaluative Analysis 

by Janet Spikes 



Ask-A-Geologist (AAG) has been in 
operation since October 1994. It began as 
an experimental Internet service offered by 
the field office of the United States 
Geological Survey (USGS) in Menlo Park, 
with the participation of several other USGS 
branches. In its first month of service, its 
35 volunteer scientists answered 127 
questions. The service has grown 
considerably since that time. In the first 
three months of 1 999, it answered on 
average 876 questions of 1 809 questions 
received each month and was able to call 
upon the expertise of 147 USGS scientists. 
Ask-A-Geologist is a relatively simple 
electronic question and answer service 
which appears to have evolved to meet 
needs as they developed, without relying 
overly much on formalized procedures and 
policies. It is now hosted by the Western 
Regional Coastal and Marine Geology 
Branch of the USGS at Menlo Park. Its 
success is testimony to the goodwill of the 
USGS volunteers who staff the operation as 
a public service. 

Information about AAG is available 
from its web site, which includes a general 
information page, Ask-A-Geologist 
Examples page. About Ask-A-Geologist 
page, and Ask-A-Geologist Disclaimers. 
Rex Sanders, the project administrator, also 
provided information in response to a 
request for general information about 
guidelines provided to volunteers, the 
service's goals and principal audience, and 
evaluation and monitoring procedures. 
Information is not available, however, on 
the web site about issues such as 
administrative policy, archiving, and 
training so that a complete analysis of the 
service based upon the Framework for 



Analysis of an Electronic Question/ 
Answering Service, v. 1.0, is not possible at 
this time. 

Mission, Objectives, Statement of 
Purpose 

The name of this service is 
Ask-A-Geologist. The name is 
straightforward and easy to remember and is 
the first step towards describing the scope of 
the service to users. There is, however, one 
element of confusion which results from 
this name. There are other electronic 
question and answer services now in 
existence that are also called 
"Ask-A-Geologist." The State Geological 
Survey of Illinois, the Geological Survey of 
Canada and services in New Zealand and 
Ohio show up on a Web search of 
"Ask-A-Geologist" as offering electronic 
question and answer services with the same 
name. It is not readily apparent which 
service is the oldest, but there is room for 
confusion among users who may have heard 
about Ask-A-Geologist. 

The URL for Ask-A-Geologist/USGS is 
http://walrus.wr.usgs.gov/ask-a-geologist. 

The mission/purpose is not clearly 
stated on the Web site. The AAG home 
page begins with the question "Do you have 
a question about volcanoes, earthquakes, 
mountains, rocks, maps, ground water, 
lakes, or rivers? You can E-mail earth 
science questions to" AAG. The disclaimer 
page does offer that the AAG "project 
provides answers to general earth science 
questions." The project coordinator listed 
three unofficial goals (i.e. not official USGS 
policy): 1) to "answer general earth science 
questions for K-12 students and the general 



public by email in a reasonable time"; 2) to 
"show the value of the USGS to the general 
public"; and 3) "to provide another, 
convenient method for USGS workers to 
interact with individuals from the general 
public, while using a broad knowledge of 
earth sciences." AAG thus provides both a 
public relations exercise for USGS and a 
public service primarily to students. 
Moreover, this field office service fits in 
well with the general mission of USGS, 
which describes itself as a science and 
information agency (see USGS main web 
site at http://www.nwrc.usgs.gov/ 
world/content/people.html): 

The U.S. Geological Survey is not 
only a scientific agency. The USGS 
is also an information agency, 
striving to make a real difference in 
people's lives by providing 
unbiased scientific observations on 
conditions ranging from the ocean 
depths to planetary frontiers. 

People, industry, and governments 
need this information to alert them 
to hazards, to help them restore the 
environment and intelligently use 
natural resources, to help them 
make sound policy decisions, and 
to improve the Nation's economy. 

Parameters of Service 

AAG answers questions about 
volcanoes, earthquakes, mountains, rocks, 
maps, ground water, lakes and rivers. On 
the basis of the three example questions 
provided at the site, it appears that answers 
are concise but complete, ranging from one 
to four paragraphs. The service does reject 
questions, apparently because it receives 
more questions than it is able to answer. 
According to the general description of the 
service, it answers about 45 percent of 
incoming messages. The About 
Ask-A-Geologist page explains that 
automated replies expressing regret are sent 
to those questioners who will not receive an 
answer because of the overload. The 



general information page also specifies that 
AAG will not write reports or answer test 
questions; will not answer questions about 
specific locations ("is my home in a 
landslide area?"); will not answer questions 
with financial impacts or recommend 
products or companies. It is not clear if 
these questions receive simple replies of 
regret or if the questioner receives an 
explanation as to why his question is 
ineligible. No other limits on the user are 
mentioned. 

Identifying the Clients 

The service is intended primarily for 
students, although nothing specific is said at 
the site other than that "we encourage 
students to send in questions." The project 
coordinator clarified that the principal 
audiences are K-12 students, the general 
public, and some undergraduate college 
students. He added that "we usually are not 
a good source of info for earth science 
professionals or graduate students." There 
are no geographic limitations on users. 

Query Form 

The user provides only a name and 
return e-mail address in addition to the 
question. The service does not conduct a 
reference interview with the questioners. 
AAG provides only an e-mail form 
accessible by clicking on "You can E-mail 
earth science questions to: 
Ask-A-Geologist@usgs.gov." It does not 
use a query form. The only guidance is in 
the form of three simple questions posted on 
its Ask-A-Geologist Examples page: "Why 
does California have so many earthquakes, 
and not New York?"; "Why is there so 
much oil in Texas, but not in Wisconsin?"; 
and "What are the deepest canyons in the 
United States?" These questions most likely 
are model questions set up by the AAG 
administrators. The same three questions 
were used in the introductory paragraph of 
the press release announcing the inception 
of AAG in 1994 (http://wwfw.esu3.kl2.ne. 
us./institute/harris/ Ask-A-Geologist.html). 
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Responsibility to the Clients 

The AAG site informs users that they 
should receive a reply in "a few days” and 
on the About Ask-A-Geologist page 
specifies that its goal is to reply to all 
questions within three working days. A nice 
"Who answers the questions?" section says 
that the questions are answered by 

USGS employees, retirees, and 
contractors who have volunteered to 
participate in the project. Most 
Ask-A-Geologist scientists work on 
these questions on their own time, 
during lunch or after work. Their job 
titles include geologist, hydrologist, 
cartographer, and technician. Their 
experience ranges from recent college 
graduates to senior scientists with more 
than 30 years experience. 

It explains that questions are not assigned 
by specialty but rather that all the volunteers 
have a general background in earth sciences, 
"and an enthusiasm for answering a broad 
range of questions." The service does not 
describe specific sources it uses to answer 
questions. 

There is another section entitled "How 
does Ask-A-Geologist work?" which 
describes in simple terms how questions are 
routed. Each volunteer receives messages 
daily routed to him by custom software. 

The volunteers are expected to research and 
answer the question within three working 
days and to respond by e-mail. 

The service does not address the 
question of user privacy. Interestingly 
enough, the above-mentioned press release 
stated that all questions and answers would 
become part of the public record and that a 
FAQ would eventually be posted. This has 
never been done so the privacy implications 
of posting questions has never been an issue 
for the service. 



The Ask-A-Geologist Disclaimers page 
states that the "answers are the best efforts 
of USGS employees, retirees, and 
contractors, and are not guaranteed to be 
correct or complete." It adds that answers 
are not the official policy of the USGS or 
the United States Government and have not 
been reviewed for correctness or 
consistency with USGS policy. 

Ease of Use 

AAG is located on the web site of the 
Western Region Coastal & Marine Geology 
branch of the USGS (http://walrus.wr.usgs. 
gov/). There is a direct and easily located 
link from the home page. However, if one 
assumes that the average user would begin 
with the home page of the USGS, the links 
are slightly more complicated. From the 
USGS home page, a user clicks on 
"Education: the Learning Web," then on 
"Adventures in the Learning Web," and 
finally on a link to AAG. Users with some 
experience with web pages would be able to 
locate it fairly easily, but novices might 
have more problems. 

AAG does include links to other 
sources. It suggests that you first search 
USGS web sites and provides a link to do 
so. It also includes links to Other Sources 
of information: USGS Internet Resources; 
USGS Learning Web; Ask an Expert (not 
maintained by USGS); and Geological 
Survey of Canada (not maintained by 
USGS). Links are also provided to the U.S. 
Department of the Interior, U.S. Geological 
Survey and the Western Coastal and Marine 
Geology Team. 

AAG's e-mail address is readily 
apparent to the user. Responses are by 
e-mail only. 

Administration 

AAG is funded by the USGS. No 
information is available at the web site 
about its funding including external funds 
from other sources, such as the National 
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Science Foundation. The service does not 
seem to be funded in such a way as to allow 
it to reach its full potential. The About Ask- 
A-Geologist page explains that software has 
been updated twice since inception of the 
service, but AAG is now turning away more 
than half the questions received. 

As a government-provided service, 

AAG is conscious of the difference between 
answers provided to school children and 
official statements of U.S. policy. The 
disclaimer cited above makes this very clear 
and also adds that "Questions requiring 
official answers from the USGS or the US 
Government should NOT be sent to 
Ask-A-Geologist." This policy does not 
seem to affect AAG’s quality or objectivity. 
It seems designed more to ensure that the 
service functions well without having to 
document every answer and archive each 
response as a public document. 

This service is free. Since its inception 
in 1994, it has been managed by Mr. Rex 
Sanders, the project administrator. The site 
does not indicate any other administrative 
staff. The questions are allocated to the 
volunteers by software. Although the 
system is automatic, it is not expert since 
questions are not distributed by subject or 
specialization but rather distributed on a 
fairly random basis to the participating 
scientists. 

There are no ethical or privacy 
statements. As mentioned before, AAG 
does not post FAQs or a bulletin board so 
has not had to confront this question. No 
public archive is maintained with names and 
e-mail addresses. No information is 
available on the web site about copyright, 
fair-use, or selective dissemination of 
information issues. No information is 
available at the web site about general 
emergency procedures or the extent of staff 
involvement in policy development. 



Staffing and Training 

USGS employees, contractors and 
retirees volunteer to answer questions. Each 
volunteer is expected to have a general 
background in the earth sciences. Age 
ranges from recent college graduates to 
long-time USGS senior scientists. AAG 
relies very much on the good will of its 
volunteers rather than on a formalized 
guidance manual. According to Mr. 

Sanders, the volunteers are sent an 
automatic reminder e-mail before receiving 
questions, which states in part, "YOU are 
the best judge of how much time and effort 
you can put into answering any question. 
You are not expected to write terms papers 
for students, nor tell oil companies the best 
places to drill. Use good judgement - you 
are a representative of the US Geological 
Survey (Personal communication, June 
1999)". He summarizes, “Most of the 
guidance can be summed up as ‘use good 
judgement’ (Personal communication, June 
1999).” Most scientists answer six 
questions once per month. It is clear, 
however, that participation is not part of 
normal work requirements. Both the web 
site and Mr. Sanders' message mention that 
most participants work on the questions on 
their own time, during lunch or after work. 

Answering Questions 

Questions are saved for 24 hours. Each 
morning, custom software distributes 
questions received the previous day to the 
volunteers. The volunteers respond directly 
to the questioner via e-mail. No 
information is given at the web site about 
personalization policies or guidelines about 
the nature of the response. The three 
examples on the web site are purely factual 
in nature. Questions are supposed to be 
answered within three business days. 

Sources for answers were not cited in the 
three examples. According to the guidelines 
provided to volunteers, they can provide 
referrals to other sources of relevant 
information. More information is not 
provided. 
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Inventory Control 

Questions accumulate for 24 hours and 
are then distributed to volunteers. More 
questions are received than AAG can cope 
with so automated replies are sent to some 
questioners expressing AAG's regret that it 
will be unable to answer the question. It is 
unclear as to whether these automated 
replies are sent only to excess questions or 
whether there is also a procedure for 
rejecting questions as inappropriate or 
beyond the scope of the service. It would 
appear that no acknowledgment letters are 
sent out, but replies should be ready within 
three working days. As stated previously, 
guidelines leave it up to the volunteers as to 
the amount of time they spend on answering 
questions. 

Coping With Demand 

There is no information at the web site 
about how AAG copes with unexpected 
demand, other than the fact that, as the 
volume of questions received grows, the 
percentage answered drops. In the first 
three months of 1999, AAG responded to 
876 questions each month at a rate of 48 
percent of questions received. This 
averages out to six questions monthly for 
their 147 volunteers. Given that volunteers 
work on questions during their own free 
time, it may not be reasonable to expect 
them to answer additional questions. Most 
likely, the acceptance rate will go up only if 
the number of volunteers increases. 

Archiving 

AAG does not maintain a public 
archive, and there is no information as to 
whether it maintains a working archive for 
administrative purposes. AAG had 
originally planned on posting a FAQ but 
never did so. 

Quality Control 

Mr. Sanders indicated they have no 
monitoring or evaluation procedures. 



Hardware/Software 

Custom software is used to route 
questions every morning to volunteers. In 
January 1998 new software was acquired to 
limit the number of questions answered 
each day and in October 1998 it was 
rewritten to allow it to save questions for 24 
hours before forwarding them to volunteers. 
According to "A brief history of 
Ask-A-Geologist" (on the About AAG 
page) this was done to enable the scientists 
to answer questions more efficiently. 
Perhaps prior to the use of this software, 
questions had been sent out on a random 
rather than scheduled basis. No information 
is provided at the site about the hardware 
being used. 

Acknowledgment 

Some automated regret letters are sent 
out, but no information about other 
acknowledgments is provided at the site. It 
seems unlikely that acknowledgment letters 
are used. Questioners must instead wait 
three business days for the reply. 

Question Negotiation 

No indication is given at the web site 
about follow-ups if the question is not clear. 

Response Guidelines 

Response guidelines are very simple. 
They provide a listing of the types of 
questions which AAG will not answer— test 
questions, information about specific 
locations, endorsements of products or 
companies, and questions based on 
information not yet released to the public. 
Other than that, volunteers are expected to 
exercise their own best judgement as to the 
nature of their reply. Mr. Sanders did not 
indicate that AAG maintains an archive of 
canned response language. Based on the 
three examples, answers tend to be concise 
and factual. However, volunteers can 
provide referrals to other sources. The 
exact form of these referrals is not known. 
The response takes the form of direct e-mail 
and is not posted on the site. The site does 
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not provide information as to whether a 
client can request follow-up or clarification. 
No policies or procedures are given at the 
site for determining priority in processing 
answers or regarding the attachment of 
non-textual materials. 

Evaluation 

No evaluation procedures are in place as 
of June 1999. AAG does maintain statistics 
and has posted some numbers on the About 
Ask-A-Geologist page. Statistics include 
monthly volume of questions, percentage of 
questions answered, and the number of 
volunteers answering questions. 

External Recognition 

In 1995, AAG won the Department of 
Interior's Innovation Award. In 1998, the 
Department awarded the Western Regional 
Coastal and Marine Geology web site, of 
which AAG is one component, a Shoemaker 
Communications Award for “its 
eye-catching, well-designed banner, team 
approach to content development, and the 
site's adept layering of information for 
multiple audiences (http://www.usgs. 
gov/98_shoemaker.html).” 

Comments 

There are a few other points which I 
would like to address which go beyond the 
scope of our Framework for Analysis of an 
Electronic Question/Answer Service, v. 1.0. 

I wanted to see how easy or difficult it was 
to find AAG. Yahooligans! and AltaVista 
both provide links to it through their 
hierarchical listing of topics. A search on 
Google! produced 6491 hits. As mentioned 
before, there are other services named "Ask 
-A-Geologist," but the vast majority of the 
hits that I looked at were related to the 
USGS service. Many were for links from 
university and education sites to AAG. I 
also found a few interesting items such as 
Bill Goffe's Resources for Economists 
(http://rfe.wustl.edu/EconFAQ.html), which 
listed AAG as "neat stuff' and a link from 
an NSF listing of science sites for children 



(www.nsfgov/od/lpa/nstw/kids/links.htm). 
Wright State University in Dayton provides 
a lesson plan built around Ask-A-Geologist 
which meets National Science Education 
Standards for Grades 5-8 (http://geology/ 
k 1 2/askg/eolog/htm) . Thus, AAG seem s 
well connected to potential client bases. 

Rex Sander's original press release 
announcing the service was on a few 
different sites. It was probably transmitted 
to many logical groups at the time the 
service began. For example, the full text is 
available still at the Educational Service 
Unit site of Omaha (http://www.esu3. 
kl2.ne.us/institute/harris/Ask-A-Geologist.h 
tml). Some sites linking to AAG describe it 
as homework help or in similar terms. 

Many of the AskA services may be 
described this way, perhaps to their chagrin. 

USGS maintains other electronic 
question and answer services, which differ 
significantly from AAG. For example, 
TerraWeb for Kids on the Western Regional 
site (http://TerraWeb.wr.usgs.gov/TRS/ 
kids/email.html) is aimed at a younger 
audience. They ask, "Want to ask a real life 
remote-sensing scientist or computer geek a 
question about remote sensing, images, 
satellites, how computers are used for 
remote sensing, or any thing else related to 
earth sciences? Would you like to share a 
story, comment, or link with us? Here's the 
place to do it!" Although the topics for 
questions are sophisticated, the web site 
presentation is definitely for younger 
children. They recommend checking with 
parents before providing an e-mail address 
and say that provision of names is not 
mandatory. TerraWeb uses a very simple 
query form. TerraWeb also provides a link 
to AAG. 

Another electronic question and answer 
service is intended for adults. The National 
Water Information Center (http://water. 
usgs.gOv/wrd005.html) "is designed to serve 
as a focus for the dissemination of 





water-resources information to all levels of 
government, academia, the private sector, 
and the general public." The electronic Q/A 
service is available by e-mail or through an 
800 telephone number. E-mail service is 
provided either to the National Water 
Information Center or directly to USGS 
state representatives and sampling stations 
for answers on local issues. 

The variety among USGS electronic 
question and answer services is interesting. 

I suspect that there is no overall USGS 
policy regarding electronic question and 
answer services, but the three services have 
different goals and audiences so one should 
not expect a standard format. 

Ask-A-Geologist is not as sophisticated 
a site as the Internet Public Library or Mad 
Scientist. However, it has certain virtues. 

Its popularity is attested to by the fact that it 
now receives more questions than it can 
deal with. Whether it can accommodate its 
popularity remains to be seen. An obvious 
project which might help reduce the number 
of unanswered questions would be to post a 
FAQ page. Whether or not AAG has the 
resources or the interest to do this is 
unclear. One especially good aspect of AAG 



is that it stresses that the electronic question 
and answer service benefits both the student 
who asks a question and the scientist who 
answers it. The About Ask-A-Geologist 
page includes the statement that "Many 
scientists enjoy finding answers to questions 
outside their specialty. Ask-A-Geologist 
scientists often learn new things and make 
new contacts that help their regular work in 
multidisciplinary projects." Mr. Sanders 
reiterated this aspect of the service; "This 
last goal was an unintended side effect of 
AAG. People 'enjoy' working on AAG, 
many doing research during lunch hours or 
after work. They like interacting with the 
general public. Many are in very 
specialized positions, and enjoy using (or 
learning) knowledge outside those 
specialties (Personal communication, June 
9, 1999)." It may be that this enthusiasm on 
the part of the volunteers will be what 
enables USGS to continue offering this 
service until such time as it may be able to 
cope with the demand. 

Sources 

Personal communication from Rex 
Sanders, Project Administrator for Ask-A- 
Geologist, June 1999. 
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Ask-A-Linguist: Evaluative Analysis 

by Margaret F. Kondashl 



Ask-A-Linguist is an online 
question/answer (Q/A) service that has been 
operating since May 1997. It is an offshoot 
of the LINGUIST List, hosted by Eastern 
Michigan University and Wayne State 
University. The LINGUIST List is a free 
service available for subscribing 
professionals to discuss linguistics research 
topics, while Ask-A-Linguist is the free 
service to which all nonprofessionals are 
directed. The questions sent to Ask-A- 
Linguist are answered by one or more 
members of a panel of volunteers from the 
LINGUIST List, and all correspondence 
with Ask-A-Linguist is kept in a public 
archive. 

Following is an evaluative analysis of 
Ask-A-Linguist using as a guideline the 
Framework for Analysis of an Electronic 
Question/Answer Service, Version 1.0, June 
4, 1999 (See Appendix A). Information 
derived from exploration of the site is in 
normal font, while critical observations 
have been bolded. 

1. 1. Mission, Objectives, Statement of 
Purpose 

1 . 1 The name of the service is Ask-A- 
Linguist. 

1.2 The URL is 

http://linguistlist.org/~ask-ling/index.html 

1 .3 The purpose of the service is stated at 
the top of the main page: 

Ask-A-Linguist is a service 
provided by The LINGUIST 
List, an Internet network for 
professional linguists. Although 
the list itself is restricted to 



messages relating to linguistic 
research, many LINGUIST List 
members are interested in 
language-related questions of 
all kinds; and a number of these 
have volunteered to staff this 
page. Ask-A-Linguist is 
designed to be a place where 
anyone interested in language 
or linguistics can ask a question 
and get the response of a panel 
of professional linguists. 

2. Parameters of Service 

2.1 As stated in 1.3, "Ask-A-Linguist is 
designed to be a place where anyone 
interested in language or linguistics can ask 
a question and get the response of a panel of 
professional linguists." 

2.2 “Because of the volume of queries, Ask- 
A-Linguist cannot answer every question. 
The following are among the topics to 
which Ask-A-Linguist does not respond: 
Translations, Etymologies, Origins of 
Phrases, Idioms and Slang Terms, Personal 
Names, English Style. If you post such a 
message, it will be deleted.” This is strong 
negative language directed at the user, 
possibly discouraging some potential 
users from posting questions. 

2.3 There do not seem to be any limitations 
on the number of questions per user. 

3. Identifying the Clients 

3.1 The specified user group is "anyone 
interested in language or linguistics." 
Logically this includes mainly 
professionals and students at the college 
and graduate levels, perhaps some 
advanced high school students. 



1 Author’s comments are in bold. 
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Linguistics is not a subject taught at 
lower educational levelS) so the service 
does not address younger users. This 
service seems to be used internationally 
by college-level students studying 
linguistics or specific language elements, 
as well as by citizenry interested in 
general linguistics topics. 

3.2 The level of expertise of the users of 
this service seems to vary but is generally 
high; almost all questions viewed were 
sent by users with some linguistics 
background. 

4. Query Form 

4.1 The form for submitting a question 
includes spaces for: Name; e-mail address; 
"what is your question about?” (which 
seems to translate into the subject line in 
the archive^; and a question box labeled 
"type your question here." 

4.2 The only way to submit a question is by 
form: no e-mail address is given for a user 
whose browser cannot support forms to send 
a question, although the address can be 
extrapolated from the archives. 

4.3 No examples or instructions for 
submitting questions are given beyond the 
guidelines on what NOT to submit (see 2.2). 

5. Responsibility to Clients 

5.1 The service indicates that the user's 
question will be forwarded to the panel of 
experts; one or more will respond by e-mail. 
No guarantee is given that every question 
will be answered, nor is it indicated if an 
unacceptable question (see 2.2) that is 
deleted is acknowledged at all. 

5.2 No information is given on how long it 
takes to get a response., Responses are given 
by a panel of experts, whose names and 
institutional affiliations are prominently 
listed on the first page of the site. Sources of 
information to answer questions are never 
mentioned, but questions seem to be 
answered from the experts' knowledge base. 

5.3 Question-answering process is 
explained only as far as "your question will 
be forwarded to the panel of experts and one 



or more will answer by e-mail." 

5.4 The query form clearly states that "In 
due course, both the question and 
response(s) will be displayed on the 
LINGUIST web site." 

5.5 There is no statement of 
liability/responsibility/appropriate use. 

6. Ease of Use/Instruction to the User 

6. 1 From the home page of the LINGUIST 
List (http://linguistlist.org) , the Ask-A- 
Linguist service is not that prominent: it is a 
text link in the middle of a paragraph of 
links entitled "Interacting with LINGUIST." 
This paragraph of links is far down the page 
and the user must scroll to get to it. 

6.2 The service is not that easy to find 
unless the user is looking for it, or unless he 
reads carefully the guidelines for acceptable 
posting to the LINGUIST List 
(http://linguistlist.org/--notice/) and 
identifies his question as out of the scope of 
the LINGUIST List. At this point he is 
directed to the Ask-A-Linguist service. 

6.3 Before he reaches the form to send a 
questions to Ask-A-Linguist, the user is 
advised to look at the past queries to the 
service and their answers, in case his 
question has been answered already. In 
addition, questions regarding the “origin of 
a word or phrase, or if it has to do with 
English style” are pointed toward a resource 
page of Online Dictionaries and Style 
Manuals, “which will answer your question 
much faster than we can.” A link to 
translation resources is also given. 

6.4 No destination address of the service is 
given to the user of the form. 

6.5 E-mail is the only delivery option for 
the response to the question. 

7. Administration 

7.1 Ask-A-Linguist is funded largely by 
subscriber donations. There are several 
places on the site where support is solicited 
unobtrusively. 

7.2 Sponsorship of the services 
(LINGUIST List and Ask-A-Linguist) by 
the two universities provides a sense of 
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authority and objectivity to the service, 
but any actual influence of the 
sponsorship (or of donations) is not 
visible, 

7.3 Ask-A-Linguist is a free service. 

7.4 The staff is loosely organized: the 
LINGUIST List is comprised of three 
moderators, who presumably moderate 
Ask-A-Linguist as well, A rather large 
panel of experts answer the questions posted 
to Ask-A-Linguist. 

7.5 Questions do not seem to be allocated 
to panel members; it appears that whoever 
replies to a given question does so of his 
own volition. According to the LINGUIST 
List moderator, “we tell them that there’s no 
obligation for each one to answer every 
question, though we hope that each question 
will receive an answer from someone 
(Personal communication, June 9, 1999).” 

8, Staffing and Training 

8.1 Volunteer subscribers from the 
LINGUIST List comprise the staff that 
answer questions. 

8.2 There is no level of expertise required 
to subscribe to the LINGUIST List from 
which the volunteers come. All the panel 
members seem to be educators; or 
otherwise affiliated with universities. 
According to the moderator of the 
LINGUIST List, the panelists are volunteers 
but most are well known linguists so their 
qualifications are not really an issue 
(Personal communication, June 9, 1999).” 

8.3 Panel members are volunteers. The 
LINGUIST List does not appear to be 
recruiting more volunteers for the Ask-A- 
Linguist service. 

8.4 There are no posted requirements for 
the volunteers regarding the answering of 
questions. Analysis of the archives shows 
that a relatively small percentage of the 
panel answers the majority of the 
questions. 

9, Answering Questions 

9.1 Based on an analysis of the archives, 
questions seem to be sent to the main e- 
mail address and then posted on some 



kind of bulletin board for the panel 
members to read and answer, A 
fortuitous And while browsing the 
archive revealed instructions sent to the 
panel members (see Figure 1), Individual 
responses are sent both to the user who 
posted the question, and to the main e- 
mail address (ask-ling@linguistlist,org) 
which seems to then post its inbox (or a 
file derived from it) as the archive. Panel 
members do not seem to know who has 
already responded to a given question, 
and consequently there is some 
duplication and contradiction in threads 
with multiple responses. Figure 1 
indicates a response from the archives. The 
bolded section addresses response 
guidelines. 

9.2 There does not seem to be any 
question negotiation, and no requests for 
further information were seen in the 
archives, 

9.3 I did not see any response that looked 
prefabricated. Based on browsing the 
archive, there does not seem to be any 
template for answering questions; the 
responses are quite varied in content, 

9.4 The name of the person answering the 
question is always given (sometimes only a 
first name). Additional personal 
information (personal e-mail, phone 
numbers, etc.) is sometimes given. There 
seems to be no effort to establish any 
personal contact nor is follow-up 
encouraged in any response. 
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Figure 1. Example of a Response in the Archive 



>Date: 

>From: 

>Subject: 

>To; 

> 

>Reply -to: 



Mon, 23 Jun 1997 10:28:52 -0400 
Ask A Linguist ~ 

Are we case-sensitive? 

Multiple recipients of list ASK-LING 
<ASK-LING@tamvm 1 .tamu.EDU> 
ask-ling@linguistlist. ORG 



>PaneI: Please remember to: 1. REPLY to the message which you got from Ask-A- 
Linguist. Don't initiate a new message. 

>2, SEND A COPY (cc) of the answer to the person who posed the 
>query. 



Query 

>From: [name and e-mail address removed] 

>Isn't it too bad that the word "whom" is dropping out of the 
>language? It's easy to use and it does make for clearer speech. 

>I for one am glad that Ernest Hemingway had read John Donne. 

>Spanish Civil War or no, “For Who The Bell Tolls” might never 
>have been a Book of the Month selection. 

> 

>Don't anthropomorphize computers... they hate that. 

Frankly, I am not going to shed many tears when "whom" finally drops out of the language— if it 
ever does. "Whom" has shown a surprising tendency to linger for generations. In any case, the 
loss of "whom" is merely the latest in a series of reductions of the English case system that 
began a millennium or more ago, a process that has resulted in the replacement of morphological 
case marking by syntactic means, most often the use of prepositional phrases, of conveying the 
same information. While you and I may miss “whom,” we demonstrate little nostalgia for 
"hwone," the accusative case form of "who," which has been long gone for centuries. Words are 
added and lost continually; people find ways of saying what they mean. 

[name and affiliation of panel respondent removed] 

Note: Bolded section addresses response guidelines. 



9.5 There do not seem to be guidelines for 
responding to questions. According to the 
LINGUIST List moderator, “the only 
instructions we give them is to be polite (not 
that many of them need this instruction) 
(Personal Communication, June 9, 1999).” 
Responses viewed include synopses of 



‘Vhat is generally known" about a 
subject, vague recollections of incomplete 
citations, solid bibliographic citations 
(sometimes including page numbers), 
referrals to other electronic services, and 
a suggestion to search the archive for the 
topic of the question (but no further 
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help/instructions). Some answers are very 
detailed; others are not. The dilTicuIty 
level is high for an electronic Q/A service, 
but the service is not aimed at children. 
The subject matter is very specialized, 
and most questions are fairly technical. 
The one question from a ninth grader 
wanted information on what sort of 
careers there are for people who are 
interested in learning languages, and 
what colleges have good language 
programs; it was unanswered! 

9.6 Based on responses in the archive, 
most are answered relatively quickly (1-3 
days) or not at all. 

9.7 Referrals are made by the panel 
members in the responses, but also the 
query page refers users to a page on the 
LINGUIST List site with links to Online 
Style Manuals and Dictionaries. 

9.8 No sources of answers given, 
sometimes source citations (of varying 
quality) given. 

10. Inventory Control 

10.1 It appears that questions 
accumulate on a first-in, first-out basis, 
but they are not necessarily answered 
that way. My guess is that they are posted 
on a bulletin board, or sent to individual 
panel members’ personal mailboxes to be 
answered as individually decided. 

10.2 There does not seem to be any 
prioritizing of questions. 

1 0.3 Response time is variable. Guidelines 
about response time are not posted. 

1 0.4 Questions that are unacceptable (see 
list in 2.2) are deleted. This is stated before 
the user gets to the query form. 

10.5 Because of the phrasing concerning 
unacceptable questions (see 2.2), I doubt 
all question are acknowledged. I did see 
the response to a question in the archive 
that could not reach the sender of the 
question (the e-mail address was wrong), 
but the panel member sent the response 
to the central list anyway, in hopes that 
the sender would see the answer there. 
This also implies that there is no 



automatic acknowledgment of questions, 
and no testing of the e-mail address 
submitted 

10.6 According to the moderator, the 
volunteers are given no guidelines 
whatsoever for responding. The answers 
are generally short and probably not 
time-consuming. Whether this indicates 
following a guadelane as aaaakaaown from 
observation. 

11. Coping with Unexpected Demand 

11.1 The only way this service discusses 
dealing with demand is to limit the types 
of questions it answers "because of the 
volume of queries." 

12. Archiving 

12.1 The archive is public: the entire text 
of every e-mail (since May 1997), including 
personal information, is kept in the archive. 
The purpose is to make available to the 
public all the questions previously answered 
by the service. 

12.2 Users are not allowed to specify if 
they do not want their questions archived. 

12.3 The archive is easily accessible from 
the Ask-A-Linguist main page. From the 
LINGUIST List main page, the user must go 
in through the Ask-A-Linguist link (the 
only link to an archive on the LINGUIST 
List main page is to the LINGUIST List 
archive, which is a bit confusing.) 

12.4 The archive is indexed by quarter 
only. The user must select a quarter to 
browse (Jan-Mar 1999, Apr-Jun 1999, etc.) 
Each e-mail is “indexed” by subject, thread, 
and date. The date index is a listing of each 
e-mail, (subject line, date and time sent, 
person from) but not in chronological order. 
Replies to each e-mail are indented. 

Clicking on subject index (a link which 
does not appear on every page, only on 
the individual e-mail message pages) 
seems to link the user back to the date 
index. The thread index is the most 
coherent. E-mail conversational "threads" 
are listed in order (although the dates of 
each E-mail cannot be seen without going 
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into each individual message), with the 
author’s name next to the subject line. Here 
it is easy to see which of the panel 
respondents answer frequently, as well as 
which questions are unanswered (the 
messages that stand alone and have no 
follow-up messages or "possible follow-up 
messages" listed after them). 

12.5 Archive is searchable by quarter or 
the entire archive. Searching capabilities are 
quite complex, allowing for the equivalent 
of Boolean AND and OR searches, phrase 
searching, truncation. The searcher can 
specify that search terms be in the same line 
and can allow for misspellings within a 
word or a “best match" function. All of 
these capabilities are wonderful to 
overcome the difflculties of natural 
language searching, but they are almost 
overkill. I think more efficient would 
have been an actual subject index to 
browse, possibly introducing a controlled 
vocabulary into the system, or at least 
organizing the existing subjects into 
categories. 

12.6 The archive does not seem to be 
selective. The notice on the query form 
implies that everything is archived, and it 
does seem to be so— even the unanswered 
questions. 

12.7 There is no FAQ file. 

12.8 According to the moderator, the 
archive is managed automatically by 
Mhonarc software (Personal 
communication, June 9, 1999). It is a static 
archive: there is no maintenance performed 
on it. An exploration of the structure of 
the archive reveals the inconsistencies of 
software that was not designed 
specifically to manage the archive of an 
electronic question and answering 
service. For example, when viewing a 
message, navigational possibilities include 
previous, subject and next subject, which 
would imply the next e-mail 
(chronologically?). Often these links are 
the same as a previous thread, next 
thread. Perhaps this is because, 
chronologically, the responses were 



posted before any other queries, but this 
is very unclear, and very confusing. The 
date index is no help in this matter; 
nowhere is there a strict chronological 
listing of postings. Also, often the 
previous, thread, next thread (which is 
useful for following all the postings on a 
topic) will link to simply the next e-mail 
(again, chronologically??), which is 
totally unrelated. I cannot believe that a 
human indexer would think this useful. 
In addition, the equivocating "possible 
follow-up messages" seems to be human 
doubt of an automated system's logic 
capabilities (e.g., following the line of 
replies to an E-mail) where in actuality 
every "possible follow-up message" I 
looked at was indeed relevant. 

13.1 Quality Control 

13.1 There does not seem to be any 
control over the quality of answers; and 
accordingly, the quality of the answers 
varies greatly. In many cases; the panel 
member did not actually answer the 
question being asked! According to the 
LINGUIST List moderator, Ask-a-Linguist 
uses a custom program written by a student 
which allows the messages to be read first 
by one of the LINGUIST list editors, then 
sent on if appropriate (Personal 
communication, June 9, 1999). 

14. Hardware/Software 

14.1 As described in 12.8, the archiving 
process is automated using Mhonarc mail- 
to-web software. 

15. Acknowledgment 

15.1 It appears that no automatic 
acknowledgment is sent after a question 
is received, and there is probably no 
acknowledgment of the deletion of 
unacceptable questions (see 10.5). 

16. Question Negotiation 

Question negotiation does not seem 
to occur at all in this service. 
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17. Response Guidelines 

Aside from the example in Figure 1, 
no guidelines seem to be available to 
panel members. 

18. Evaluation 

It does not appear that any 
evaluation of answers, of the 
contributions of panel members, or of the 
service as a whole is occurring 

The three moderators for the 
LINGUIST List, who seem also to have 
Ask-A-Linguist under their jurisdiction, 
obviously have a huge responsibility. The 
above flndings, while revealing seemingly 
careless practices for a Q/A service, make 
sense in light of the short stafflng and 
reliance on donations: reliance on 
volunteers to answer questions; no 
requirements of volunteers; no quality 
control of answers; no assurance that all 
questions will be answered; no value- 
added indexing; possible automation of 
archive. This critical analysis is not 
meant solely to reveal the flaws of this 
Q/A service but also to show one area 
where AskA services are not well- 
developed and maintained. Ask-A- 
Linguist is the only (or one of very few) 
free service available that addresses 
questions of this nature for a mainly 
adult audience and is thus not really 
comparable to the K-12 science Q/A 
services. There is not much AskA service 
coverage in this area, and it may be that 
there is not much demand for linguistics 
Q/A services because linguistics is not 
taught at lower levels of education. 

Follow-up questions concerning 
quality control, response guidelines, and 
archiving were sent to the moderators of the 
LINGUIST List, as well as to the central 
e-mail address for the service. The 
moderator’s replies are incorporated at 
pertinent sections. 



Sources 

Personal communication from Helen 
Dry, Moderator, LINGUIST List (June 9, 
1999). 
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Ask A NASA Scientist!: Evaluative Analysis 

by Teresa L. Aquino 



This paper is a case study of the Ask A 
NASA Scientist! (http:/imagine.gsfc.nasa.gov/ 
docs/ask_astro/ask_an_astronomer.html) 
question and answer service according to the 
subdivisions of the Framework for Analysis of 
an Electronic Question/Answer Service Version 
1 .0 (June 4, 1999). It also contains a conclusion 
in which some general observations of the 
service are articulated. 

Mission, Objectives, Statement of Purpose 

The name of this service is Ask a NASA 
Scientist!, and it is located at http://imagine.gsf. 
nasa.gov/docs/ask_astro/ask_an_astronomer. 
html. The parent organization is NASA, and this 
page is part of the Imagine the Universe! web 
site, a service of NASA's High-Energy 
Astrophysics Science Archive Research Center 
(HEASARC) within the Laboratory for High 
Energy Astrophysics (LHEA) at NASA's 
Goddard Space Flight Center. 

The name Ask a NASA Scientist! is a true 
statement, but it is a little misleading. It could 
be misconstrued as referring to a very large pool 
of any of NASA's scientists, but this is not the 
case. The true purpose of the service, which is 
not readily apparent from either the Imagine the 
Universe! site or the Ask a NASA Scientist! 
homepage, is to answer questions from people 
ages 1 5 and up related specifically to X-ray, 
gamma-ray, and cosmic-ray astrophysics. A link 
from the Ask a NASA Scientist! homepage 
called “Who are we and what kind of questions 
do we specialize in?” gives this purpose as well 
as other useful information. The HEASARC 
has its own web site, at http://heasarc.gsfc.nasa. 
gov/docs/outreach. html. This site has a link to 
Ask a NASA Scientist! 

Parameters of Service 

This service answers only unique questions. 



It will reject questions for any number of 
reasons, including: non-questions, invalid return 
e-mail addresses, homework questions, 
questions outside of the scientist's scope, 
obvious laziness on the part of the user, or if the 
scientists lack time to answer the question. The 
one stated limitation on using the service is that 
users submit only one question per week. 

The service also has a link to “Guidelines for 
Teachers” in which teachers are asked not to 
overwhelm the service by making writing to it a 
class assignment. Instead, they suggest that 
teachers use the information that is already 
available on the Imagine the Universe! web site. 

Identifying the Clients 

According to the service homepage, the 
intended users are people ages 1 5 and up who 
have serious interests and questions about 
astrophysics. Children ages 4-14 are directed to 
StarChild, a different NASA service. But in 
contradiction to that statement, the HEASARC 
homepage says that users of any age may ask 
questions. Considering that the answers given 
to questions usually require a very high level of 
science and reading comprehension, the 
statement on the HEASARC page should 
probably be revised. 

Query Form 

The information required to ask a question 
can be found on the same page as the web-form. 
The user must provide only a name, an e-mail 
address, and a question. In the directions at the 
top of the form the service requests that the user 
indicate within the question a grade level (if in 
school) or familiarity with astronomy. The form 
is the only means of sending a question on this 
page, and there is no guidance or examples 
given on how to state a question. An e-mail 
address exists for the Q/A service in Search the 



28 




34 



Archives; but again the address is not evident on 
the web-form page. 

Responsibility to the Clients 

The service indicates that it will provide an 
e-mail response to the user within one to three 
weeks. But it also emphasizes that the scientists 
are really busy, that everyone is a volunteer, that 
they receive hundreds of questions per week, 
and that they cannot answer every question. It 
also takes every opportunity to suggest that 
users look first in the achives to see if their 
question has already been answered. In all of 
these ways, it absolves itself of the 
responsibility for answering questions in a 
timely manner or at all. 

It does not indicate in what form the 
response will be provided or what sources will 
be used. It appears, however, that the questions 
are answered by the volunteer scientists who are 
themselves experts in this field. There is a link 
to information on the staff on the Imagine 
homepage. The site does not explain the Q/A 
process, user privacy, or liability. There are no 
indications either of what the answer will look 
like. The only way to find that out is to look at 
previously answered questions in the Archive. 

Ease of Use/Instructions to User 

It takes four clicks to get to the web form. 
The Ask a NASA Scientist! link is in the form 
of the third navigational button from the left on 
the bottom of the Imagine home page. In fact, it 
is in the same place on every page within the 
Imagine site which makes for very easy 
navigation throughout the site. The service is 
not, however, mentioned elsewhere on the 
homepage. 

Once there, the Ask'a NASA Scientist! 
homepage frequently directs users to other 
sources of information including: StarChild (a 
NASA service for young readers), the web, the 
Ask a NASA Scientist! Archive, and its own 
Hot Topics, Special Features, and Primary 
Areas of Interest and Expertise. At the very 



bottom of this list is the link to "Submit Your 
Question." 

Once a user clicks on "Submit Your 
Question" they are brought to another page 
called “Guidelines for Asking Your Question” 
which provides a list of do's and don'ts. Users 
can then click on the "Ask a NASA Scientist" 
button at the bottom of the page, which will then 
bring them to "The Ask A NASA Scientist Post 
Office." 

The destination address is not apparent to the 
user of the query form. However, if you do a 
search in the achives, the results generated will 
bring up a link to Ask a NASA Scientist! that 
includes an e-mail address (astro@heasarc.gsfc. 
nasa.gov). The only form of response a user will 
receive will be by e-mail-there are no other 
options, and the service specifically states no 
responses are provided by regular mail. 

Administration 

This service provides no written statements 
on how the service is administered. 

In terms of resources, there is no statement 
on how the service is funded, but, as it is part of 
the larger Imagine site, it would appear that 
NASA is the sponsor (at least for server space). 
The service is free and open to all. There is no 
indication of how staff is organized other than 
that there are eighteen web site text authors who 
make up The Imagine! Team, including a 
project leader, and a technical rep. There are 
also no procedures for allocating questions, no 
ethical statements, no privacy statements 
(although questions in the achives are stripped 
of any identifying material), and no indication 
of copyright, liability disclaimers, or emergency 
procedures. There is also no indication of any 
kind of policies at all that are available to the 
public. 
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Staffing and Training 

A small number of staff members (either 
astronomers or programmers) from HEASARC 
volunteer to answer questions. There is no 
indication about how the volunteers are 
recruited, trained, or assigned responsibilities. 

Answering Questions 

The site does not give any information as to 
how questions are processed. Questions are sent 
using the online form (or perhaps e-mail, if the 
user sees the address) and then answers are 
returned via e-mail. It does not appear that any 
question negotiation occurs; questions are either 
accepted and answered or rejected. Answers are 
not very personalized— there is no greeting 
usually, and the common sign-off says "I hope 
this helps” with the name of the answerer 
following. No instructions for follow up are 
given to the user. 

If there are guidelines for how answerers 
should answer a question, they are not published 
on this site. Turn-around time is stated as 1-3 
weeks. There are no written procedures for 
referrals or citing sources either, although 
several answerers do include links to other web 
pages as part of their answers. 

Inventory^ Control 

There are no written guidelines posted on 
how questions are queued, assigned, prioritized, 
or selected by answerers. Again response time is 
stated to the users as 1-3 weeks, or perhaps 
never. When a question is sent to the service, a 
confirmation screen appears to let the user know 
their question has been sent, but again it 
includes caveats on how long it will take to be 
answered if it is answered at all. Finally, there 
does not appear to be a^numbering or dating 
system either; how questions are tracked is 
unknown. 

Coping with Unexpected Demand 

There are no written guidelines on how the 
service deals with unexpected demand. From 
what can be seen, it is already overwhelmed 
with questions and is expending a lot of energy 



to encourage users to check the archives and 
other sources of information before submitting 
questions. It also clearly states that it cannot 
answer all the questions submitted. 

Archiving 

The Archive is the one positive part of the 
whole service. Questions are stripped of all 
identifying information and then are usually 
answered in a very detailed and thoughtful 
manner. They are sorted by subject into 
astronomy-related categories, including a Hot 
Topics section which is similar to an FAQ. The 
achives are freely available to anyone and are 
keyword searchable. Unfortunately the items are 
not dated, so it is impossible to tell the age of a 
question/answer. It appears that the Archive is 
selective in what it posts, although without 
documentation this is impossible to confirm. It 
appears that the preference is to post answers to 
complex or unique questions, i.e., those which 
cannot be answered by anyone else or cannot be 
located anywhere else on NASA's web sites. 
Who manages the Archive is not clearly stated, 
although one can assume it is the Imagine team. 

Quality Control 

How quality control is maintained is 
unknown, as no written procedures are included 
on the site. 

Hardware/Software 

The hardware and software are not indicated 
either, although from observation the site is web 
based (there is not a text only option), and it has 
a search engine for searching the archives. 

Acknowledgment 

As mentioned before, a response screen is 
automatically generated immediately after a 
question is submitted via the form. It contains 
the generic "don't expect much right away" 
statement, and then it shows a copy of the 
question, which was submitted. 

Question Negotiation 

There are no instructions on how to proceed 
with question clarification. 
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Response Guidelines 

It appears that responses are generated one at 
a time, by an individual or groups of people 
working together, depending on the area of 
expertise required. Some responses are short, 
others are lengthy, and all are written at a very 
high-level, even when the user indicates a lower 
grade level. Responses are e-mailed directly to 
the user and at some point some are posted to 
the Archive. It is not clear if all questions 
answered are posted to the Archive. 

Due to the lack of written policy 
documentation on the site, it is impossible to 
know what if any guidelines are used when 
responding to questions. 

Evaluation 

It is unknown at this time if the service 
conducts any kind of formal or informal 
evaluation of itself. There is no indication of it 
on the web site. 

External Recognition 

A search on the site, and on Yahoo, Lycos, 
the EBSCO database, and the ScienceNews web 
site was unsuccessful in locating any 
information regarding awards that the site has 
received or articles that have been written about 
it. 

Conclusion 

This site is still a work in progress. It is 
staffed by volunteers who love the work they 
do, and who want to share their enthusiasm and 
knowledge of this particular area of astronomy 
with anyone else who is interested. 

While the actual high-level content provided 
in the answers to questions is very good, and the 
organization of the topics in the archives is 
excellent, the service provides less than 
satisfactory information about its administrative 
policies and procedures. There is no 
documentation on the site as to how the service 
is managed, how questions are to be answered, 
or how volunteers are trained. 



From all the evidence, it appears that this 
service was pulled together in a rather ad hoc 
manner. Because of poor initial planning it has 
since run into the problem of how to continue to 
provide service in the face of overwhelming 
numbers of questions. In response to this level 
of demand, the service has chosen to be very 
selective in the kinds of questions it answers 
and to help users locate the information they 
seek using other sources of published 
information on the web. 

Being selective has its drawbacks in terms of 
customer service. Users never know if or when 
they will receive a reply to their questions. This 
can have an effect on user return rate. In this 
case, long turnaround time is clearly being used 
as a deterrent to questions. 

Overall it is a site that contains very good 
information; however, the Q/A service itself 
could stand some improvements, particularly in 
how its administration and policies are 
articulated to users and in how quickly answers 
are provided. 
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Ask Dr. Math: Evaluative Analysis 

by Laura W, Speer 



1. Mission, Objectives, Statement of 
Purpose 

1 . 1 What is the name of the service? 

Ask Dr. Math 

1.2 What is the URL? 
http://forum.swarthmore.edu/dr.math/ 
ask.html 

1 .3 What is the mission/purpose of this 
service? 

The mission is to answer questions 
about mathematics from K-12 students and 
their teachers.. 

1.4 Is the mission/purpose apparent to the 
user? 

Yes, the mission is easy to discern. 

2. Parameters of Service 

2.1 What types of questions does this 
service answer? 

Ask Dr. Math answers all types of 
questions— short and long answer. 

2.2 Does the service reject questions? 

If so, what is the basis of rejection? 
They do not answer all questions 

because of the volume they receive. They 
also test e-mail addresses and do not answer 
those that bounce back (duds). 

2.3 What limits exist on the service for an 
individual user? 

The only limit apparent on the service 
is a set of instructions to teachers about 
appropriate classroom use of the service. 

3. Identifying the Clients 

3 . 1 Who are the intended users? 

The service states that it answers 
questions from K-12 students and their 
teachers - no distinction is made other than 
that. There is an area for questions at a 
higher level and some of these were from 
college students. It also states that generally 
English is the only language that can be 
used to pose questions. 



4 . Query Form 

4.1 What information is required to 
answer a question? 

Name; age; e-mail address (full 
address, no mistakes); subject (be specific); 
question 

4.2 What options are available? 

A form is given as first option; an 
e-mail address is also given. 

4.3 What guidance is given on how to ask 
a question? 

“Tell us how you’ve tried to answer it 
yourself; if stuck let us know where so we 
can help without doing parts you've already 
figured out.” 

5* Responsibility to the Clients 

5.1 What does the service indicate it will 
provide to the client? 

Assistance in performing a math 
function. 

5.2 Does it inform the client about: 

- how soon the question will be 
returned? 

The goal for questions answered is 
1-2 days. 

- who answers the questions? 

A page explains who answers the 
questions, and another page lists all of the 
"Drs." names with pictures and web sites (if 
they choose to provide them). 

- The types /specific sources used? 
There is a list of suggested sources, 

both printed and electronic. 

5.3 Does it explain the question 
answering process? 

Yes, in the Forum Doctor's Office. 

5.4 Does the service address user 
privacy? 

Sort of. 

How? 

The math forum has a web page that 
addresses children and the Internet, and in 




32 

38 



the description of the service (in the 
Doctor’s office area). There is no privacy 
statement on the query form. According to 
the service administrator, the service does 
not have an official privacy statement; 
however, they do not distribute any of the 
names or e-mail addresses of questioners to 
outside sources. In house the only use of 
this information is to contact the user for 
purposes of gathering information about the 
service (evaluation). At present the names 
and e-mail addresses do appear in the 
archives, but they are in the process of 
removing them. 

5.5 Is there a statement of liability/ 
responsibility/appropriate use? 

Yes. 

5.6 What is included in it? 

It gives examples of appropriate use 
on the Write to Dr. Math page— homework, 
puzzles, math contest problems or any math 
topic. 

6. Ease of Use 

6.1 How is the service positioned on the 
web page? 

The link is on the top of the forum 
features list. 

6.2 Is it easy or hard to find? 

It is one link from the Math Forum 
home page. 

6.3 Does it direct you to other sources? 
Yes, FAQs and links (directly); there 

are also links to other areas in the math 
forum, links to other sites, and print source 
bibliographies. 

6.4 Is the destination address apparent to 
the user of a form query? 

No, but there is a place on the web 
site that provides it. 

6.5 What options are available? 

The answer is e-mailed to the 

questioner and also placed in the public 
archive. 

7. Administration 

7. 1 How is the service funded? 

The service is funded through a 
combination of a grant funded by the 



National Science Foundation and support 
from Swarthmore College, Hewlett Packard, 
and Microsoft. It is a service that is part of 
the Math Forum, which has partnerships 
with many other organizations, such as the 
Eisenhower National Clearinghouse for 
Mathematics and Science Education and the 
National Council of Teachers of 
Mathematics (NCTM). 

7.2 How does funding/sponsorship 
influence quality and/or objectivity? 

It does not. 

7.3 Is service fee-based? 

Not at this time, but may be for some 
users in the future. 

7.4 How is staff organized? 

The staff is a combination of paid 
employees and volunteers. 

7.5 Who allocates questions to 
answerers? 

It is done manually. At present 
answerers (doctors) can self-select or 
administrators (nurses) can assign. 

7.6 Are there ethical statements? 

Are they easily publicly accessible? 
No. 

7.7 Are there privacy statements? 

Are they easily accessible? 

There is a page that addresses 

children on the web and filtering, but not 
privacy directly. (See Section 5.4). 

7.8 How are copyright, fair-use, selective 
dissemination of information issues 
managed? 

7.9 Are there liability disclaimers and/or 
content disclaimers? 

No, accuracy is monitored both in the 
question-answering and archiving of 
questions. 

7. 1 0 Are there emergency procedures? 
There is a section on software and 

software support, but in the online 
documentation overload or other emergency 
procedures are not addressed. Ask Dr. Math 
does discourage teachers from inundating 
the service with a set of class questions, 
because it will "stress” the system. 

7.1 1 To what level are staff involved in 
policy development? 



O 

ERIC 



39 33 



From the documentation that is 
available online it is apparent that staff 
input is solicited continually through 
planning meetings, staff newsletters (this is 
new), and the "community center" 
electronic support area. 

8. Staffing and Training 

8.1 Who will answer questions? 

Volunteer and staff "doctors" answer 

questions. 

8.2 What level of expertise is required? 

It varies according to the difficulty of 

the question. 

8.3 How are answerers recruited and 
selected? 

There is a form on the Internet that a 
person wishing to volunteer fills out and 
lists their areas of expertise. There have 
also been a few test cases of "doctors" 
recruiting teams from their workplaces. 
According to the service administrator, 
anyone who has communicated with the 
service about the possibility of becoming a 
volunteer is sent a response that includes the 
URL for an area of the service that explains 
the answering procedures in more depth. 

The different areas of the service are 
explained, and interested persons are able to 
view the actual questions that are going 
through the process. If they are still 
interested in volunteering after this preview, 
they are asked to fill out an web form that 
includes name, e-mail address, 
college/year/major (if applicable, and a 
block to include information about 
themselves, strengths they feel they have, 
and any questions they may have. 

8.4 What is expected/average time 
commitment for answerers? 

Whatever time a person can volunteer 
is acceptable. There are no apparent 
statistics on average time apparent on the 
web site. 

8.5 What training exists for the answerer? 
"When new Doctors receive their accounts, 
they undergo a screening process. During 
this screening, responses that they write to 
students do not go directly to the students. 



but rather they sit in the Holding Tank 
(currently the Holding Tank is located at the 
top of the Triage Page) awaiting approval 
by the administrative staff. When the 
administrators review a new doctor's work, 
they are given the chance to edit the doctor's 
response (although generally administrators 
ask the new Doctor to rework the answer 
him/herself to facilitate the Doctor's 
training), to send comments via e-mail to 
the new Doctor, and to write an entry in the 
new Doctor's evaluation file. This 
evaluation file is crucial to the 
administrators when they make decisions 
about whether to promote a Doctor from 
"new" status to "tenured" status. The 
Doctor's Office keeps track of who answers 
which questions, and, in the case that the 
question went through the Holding Tank, 
the member of the administrative staff who 
approved the question is also noted (from 
The Doctor's Office System Functions).” 
According to the service administrator, new 
volunteers are provided with many types of 
guidance. They are given the URL of pages 
that guide them in answering questions, are 
monitored by more experienced doctors as 
was already described, are encouraged to 
explore the archives and FAQs as resources. 
They also have access to a MEMO area 
where they may post questions they have to 
other Doctors asking for help and receive a 
monthly electronic newsletter. Tips on 
writing good responses are included in the 
monthly newsletter, such as "In order to 
give proper and thorough information to the 
students, our answers are sometimes long 
and/or technical. Please be sure that when 
you write long or technical answers, you 
emphasize clarity in your writing style. In 
addition, be sure to encourage the student 
to write back for more help, or point out that 
the material is difficult. This way, if the 
student does not understand, we hope he or 
she will not become discouraged 
(Newsletter, May 1999).” 

8.6 Are there seniority benefits and/or 
increased responsibilities? 

Yes, when a volunteer is deemed 
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tenured, he can then send out answers with 
having them checked and if he is chosen to 
become an administrator, he can do even 
more, e.g., change doctor records, move 
things to the archive. The third high-level 
clearance functions within the archiving of 
questions. 

8.7 If answerers are staff, how does the 
Q/A service responsibilities fit into their 
other responsibilities? 

Some of the administration/editing/ 

archiving is handled by them 
according to their positions; answering by 
staff is not addressed. 

9. Answering Questions 

9. 1 Describe the process a question goes 
through. 

A question is received; given an id#; 
subject area is identified (either by expert or 
human system); an e-mail is sent to 
acknowledge receipt of the question by the 
service; the question is put into the queue 
where answerers can choose question or 
nurses can assign questions; doctors answer 
a question, e-mail the result to the patient, 
and mark the question complete and ready 
to be archived. 

9.2 Are there question negotiation 
guidelines? A incoming message that is not 
a new question but rather a continuation of 
an exchange between doctor and patient can 
be added to an existing thread, allowing the 
exchange to be recorded as a single visit. 

Is communication through other 
media an option? 

Yes, an address and phone number are 
available on the Math Forum site. 

9.3 What templates/canned answers 
exists? 

The question form and comment form 
are web forms; the response sent to 
acknowledge receipt of the letter is canned. 
There may be others not known at this time. 
According to the service administrator, 
volunteers are guided in preparing thorough 
and comprehensible responses. There are 
some standard responses, but in the area of 
question-answering the questions are unique 



and the answers are personalized. Templates 
are not used, but some of the information is 
provided automatically by the software, i.e. 
name of questioner, e-mail address, and 
question. 

9.4 How much personalization is given? 
The answers all seem to be 

personalized. 

9.5 Are personal contacts to answerers 
given in responses? 

Not generally, but there may be a few. 
Their names and sometimes more 
information is available to the public. 

9.6 What guidelines exist about the nature 
of the response? 

This is not available online. 

9.7 What guidelines exist for response 
time, if any? 

1-2 days; difficult questions may be 
longer. 

9.8 What is the policy and procedure for 
referring questions? 

All doctors may send memos asking 
for help. 

To whom are referrals made? 
"Tenured" (more experienced) 
doctors. 

9.9 Are sources of answers given? 

If cited, yes; if just an explanation is 
given, no. 

10. Inventory Control 

10.1 How do questions accumulate? 
Questions accumulate in what is 

called the triage section. It is a queue. Are 
questions assigned? 

Some are assigned by the "nursing 
staff (administration), and some are chosen 
by doctors (answerers). 

10.2 How are they prioritized? 

Last in, first in line. 

10.3 What guidelines are given for 
response? 

1-2 days. 

10.4 What is done with questions or clients 
not matching guidelines? 

It is acceptable to take longer for any 
questions that requires a more lengthy 
answer, or it may take longer for someone 
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to choose it. 

10.5 Are all communications 
acknowledged? 

Yes, automatically by the e-mail 
server. 

1 0.6 What are the guidelines on time limit? 
Not available online. 

11. Coping with Unexpected Demand 

11.1 Are there scheduled down-times? 

Not announced online. 

1 1 .2 How are down-times communicated 
to users? 

No one is guaranteed an answer and 
that is explained on the question web form 
page. 

1 1 .3 What is done if system is inundated? 
Many of the questions will not be 

answered. Also, it may cause the system to 
malfunction (explained on the not 
acceptable use page for teachers). 

1 1 .4 Is there a limit to the number of 
questions the system can handle? 

Yes, I think so, but it is not revealed. 

12. Archiving 

12.1 What is the nature of the archive? 

It is a public archive. Material that is 
of general interest or is unique is included. 
The archive is fluid; it is continually being 
updated. The e-mail address/name of the 
questioner and the doctor’s name are 
included in the archive (no anonymity). 

12.2 Are users allowed to specify if they 
do not want their question archived? No. 

12.3 How easily accessible is it? 

Very easy. 

12.4 Is it selective? 

Yes. 

Is it indexed or searchable? 

Yes, it is searchable and organized 
both by grade level and math discipline. 

12.5 Is the archive selective? 

Yes, only unique or general interest 
questions are included. 

12.6 Is there a FAQ? 

Yes, and it is organized and 
searchable (by grade level and discipline). 

12.7 How is the FAQ related to archive? 



The FAQs are articles that explain 
certain math problems, terms, and formulas 
that many students find useful. 

1 2.8 Who manages the archive? 

The archive is managed by the Ask 
Dr. Math Doctor’s Office Staff. 

13. Quality Control 

13.1 How is quality control maintained? 
There are resources available on the 

math forum web site; doctors may confer 
with other doctors (electronically); and, in 
some instances in the future, someone may 
add further information to an answer to 
further clarify it. 

13.2 What happens if quality is 
inadequate? 

Please refer to the training of staff in 
Section 8. They are asked to rewrite the 
response (with guidance). 

13.4 Is there a review process for new 
answerers? 

Yes. 

[13.5] How does the system respond to 
negative feedback? 

According to the service 
administrator, any communication from 
patrons, whether positive or negative, is 
responded to. They get very little negative 
feedback. 

14. Hardware/Software 

14.1 Describe the hardware/software used 
by this system. 

The Forum Software Department web 
page (http://forum.swarthmore.edu/tools/) 
describes the software used to design and 
run the site. There is a schematic drawing 
of the building that lists Ethernet and 
telephone jacks, powerbooks, and NT 
machines. Math Forum thanks Hewlett 
Packard for their support, so one can 
assume that they have at least some Hewlett 
Packard Hardware. The Doctor's Office 
software is a set of programs designed to 
specifically run this service and is 
accessible to anyone having Netscape 2.0 or 
higher online. 
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15. Acknowledgment 

15.1 What kind of acknowledgment is 
sent? 

A ’’canned" response. 

15.2 How soon is it sent? Within minutes. 

15.3 What information does it contain? 

It thanks the user for sending in his 

question (does not restate it) and gives 
suggested places to look for information 
while waiting for the question to be 
answered 

15.4 Is it sent automatically? 

Yes. 

16. Question Negotiation 

16.1 If question requires clarification, what 
are the guidelines? 

Question negotiation is not really 
addressed (or done). If a person responds 
with another question about the first query, 
it is added to the first query in the archive. 

17. Response Guidelines 

1 7. 1 How is the response generated? 

This is not addressed. It appears to be 
mostly free form, but a request for 
information has been sent. 

1 7.3 Limitations or constraints on the 
answers? 

None that are apparent in the 
documentation. 

1 7.4 What documentation accompanies the 
answer? 

If a source is cited, it will be listed; 
many of the answers are just explanations 
with no sources given. 

1 7.5 How does the service respond to 
ineligible/inappropriate questions? 

The answer to this is not apparent. 
There is, however, instruction about what is 
appropriate use of the site. 

1 7.6 How does the service address level of 
knowledge/ language skills of the user? 

The answers are categorized by grade 
level, and answers in that area should be at 
that level. 

17.7 What policy restrictions are there on 
sources? 

Not addressed. 



17.8 Are previous answer available and 
searchable? 

Not addressed. 

1 7.9 What format does the response take? 
An e-mail is sent to the questioner, 

and questions of general interest are posted 
on the archive. 

17.10 How does the questioner request 
clarification of the answer? 

By replying to the answerer. 

17.1 1 How does the client request 
follow-up? 

Same as 17.10. 

17.12 Is there a policy on response time? 

1-2 days generally. 

17.13 Is there protocol for determining 
priority of messages? 

A staff member (nurse) monitors this 
queue and assigns questions that are not 
being selected. 

17.14 What are policies regarding attaching 
to non-textual materials? 

Although not addressed in the 
documentation, attachments of web sites are 
used frequently. 

18. Evaluation 

18.1 Is there a formal evaluation in place? 
Yes. 

1 8.2 What is the nature of the procedure? 
There is a comment form; you may 

reply to the answer sent you; and in Spring 
1999 a formal survey was done online. 

1 8.3 How frequently does it occur? 

Not addressed. 

18.4 What measures of performance are 
used? 

Although the only items shared online 
are the thank you notes, other statistics are 
gathered but are not available to the public. 

1 8.5 Is there a follow up to the user? 

Not addressed. 

18.6 How does the service evaluate the 
quality of answers provided? 

See 8.5. 

18.7 Does the service meet mission goals? 
Yes. 

1 8.8 What statistics are maintained by the 
service regularly? 
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Statistics mentioned by the service 
administrator: number of incoming 
questions; number of answered questions; 
percent answered. These may not be all. 
How are they used? 

Statistics are use in the writing of 
grant proposals, management guidance, and 
reports to the grant organizations that 
support the service. According to the service 
administrator, statistics are kept for use in 
evaluation of the service. Some of the 
statistics (number of incoming 
questions/number of answered 
questions/percent answered) are shared with 
the volunteers in the monthly newsletter. 

19. External Recognition 

19.1 What awards has this site received? 
1999 Webby nominee; 1998 Virtual 

Reference Desk (VRD) Top Six Award; 
many others are listed on the web site 
(http://forum.swarthmore.edu/forum.awards 
.html). 

19.2 What articles are written about it? 

At least 30 newspaper and magazine 

articles recommended it as very useful for 
parents and teachers who need guidance. 

No scholarly articles were located. 

Comments 

The questions/concems that were 
directed to the service for clarification, are: 

1 . Are canned responses used? 

2. How is privacy addressed? The names 
of children (and contact information) is 
available publicly online. 

3. Although the option of e-mailing in a 
question if you cannot use the web form is 
offered, the e-mail address was not available 
on the Ask A Question page. 

4. What is the experience level that they 
are seeking in volunteers? 

5. Does the service receive any negative 
feedback and what do they do with it? 

6. What type of statistics do they gather? 

7. Are templates used by Doctors when 
answering questions? 



The service administrator’s responses have 
been incorporated into the text. 

The Ask Dr. Math question and answer 
service is an excellent choice to use as a 
model for designing a new service. The 
planning and management of the service is 
apparent in the in-depth documentation that 
is available to the casual patron or the 
interested information professional. Ask 
Dr. Math’s attention to detail and dedication 
to math education is to be commended. I 
was surprised and impressed by the number 
of professions and organizations that were 
involved in the planning, launching, and 
continued support of Ask Dr. Math. Any 
organization that is considering planning a 
questions and answer service should keep in 
mind that it will be more successful if more 
people (professional, community, and 
organizations) are allowed/invited to 
participate in the original/future planning 
and subsequent evaluation. 

Sources 

Personal communication with the Ask 
Dr. Math service administrator, June 1999. 
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Ask Dr. Universe: Evaluative Analysis 

by Kristen Gramer 



Mission, Objectives, Statement of 
Purpose 

Ask Dr. Universe (http://www.wsu.edu/ 
DrUniverse) is an apt name for this question 
and answer (Q/A) service. The site, which 
is operated by the Graduate School and 
News and Information Services, 

Washington State University (WSU), 
provides answers to questions on any topic a 
user is curious about. The mission of the 
site is to use Dr. Universe s resources to 
answer the questions that people have and 
are either afraid to ask or do not have the 
proper resources to find the answers to on 
their own. Dr. Universe answers questions 
on all topics from science to literature to 
philosophy and beyond. The question 
“Why Dr. Universe?” is answered within 
the web site. From the main contents page, 
the user can select “Who is Dr. Universe?” 
then easily go to “Why Dr. Universe?” from 
there. 

Parameters of Service 

There are no guidelines on the types of 
questions that the service answers. The 
questions can be on any topic. There is no 
mention of appropriate question length. The 
query form has a Question box that 
measures 4 x 1 inches. Most of the 
questions in their archive range from one 
sentence to a short paragraph. The service 
has a question volume of approximately 600 
questions per week. Time constraints only 
permit the staff to answer 1 5-20 percent of 
the questions submitted. The service hopes 
to alleviate this problem in the future by 
increasing the number of staff. There do not 
appear to be limits on the use of the service 
for individual users. Within the questions 
included in their archive, several of the 
users identify themselves as repeat users. 



and there are many queries that contain 
more than one question, and, at times, 
several unrelated questions at once. 

Identifying the Clients 

The web site itself does not specify a 
specific user group other than people who 
have questions they would like to have 
answered. The service is extremely 
kid-friendly. Dr. Universe is depicted as a 
large cartoon cat and other graphics on the 
various pages include cartoon images of 
spiders, ladybugs, bees, etc. The general 
tone of the text is also kid-friendly. A child 
may not understand every word he sees, but 
he will not be discouraged from using the 
site by the few words he encounters that he 
may not know. 

Query Form 

The user is required to provide very 
little information when they submit a 
question. The user must write his question 
and e-mail address in the spaces provided. 
The form solicits other optional 
information: 

1 . First name, last name. 

2. Age. 

3. City; state. 

4. Gender (from pick list). 

5. Academic status: student or teacher. 

6. Grade level: not a student, grade school, 
middle, high school, college, faculty - 
elementary or secondary, faculty - 
Collegiate. 

7. Source of information about Dr. 
Universe (from pick list: friend, teacher, 
family member, WSU web site, another 
web site, newspaper, other). 

8. If web site, the URL. 

9. If newspaper, its name. 
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10. Anything else you want Dr. Universe to 

know about you. 

The optional information is used by the Dr. 
Universe staff to identify characteristics of 
the users and to determine the appropriate 
level of response to the user’s question. 

The user can submit a question using 
the query form. The user also has the 
option of e-mailing Dr. Universe directly 
and a regular mail address is also provided. 
The site does not indicate how it will 
respond to questions submitted by regular 
mail. Questions submitted electronically 
will be answered by e-mail and posted to 
the archive. When the user selects the query 
form, there are no specific guidelines about 
the type of question or the form the question 
should take. The only instructions are for 
the user to ask about something he has 
always wondered about. There are several 
places in the web site where the user can 
access previously asked questions to 
develop a sense of the types of questions 
that have been asked or to see if his own 
question has already been answered. There 
is a Featured Question which includes a 
longer than usual response to a user’s 
question from a WSU faculty member. 
There is also Today’s Question, which 
changes daily, and a searchable archive of 
previously asked questions. 

Responsibility to the Clients 

Dr. Universe pledges to get the answers 
to users questions. The questions will be 
answered by researchers at WSU, if 
necessary. One service that will be 
provided, if requested, is to put the user in 
touch with a specific faculty member. On 
the Contents page. Dr. Universe says “If 
you'd like to contact any of the faculty 
members who have answered questions for 
Dr. Universe about their research, 
scholarship or creative work, just e-mail the 
Dr. — she'll put you in touch with the person 
behind the answer! 



(http://www.wsu.edu/DrUniverse/Contents. 
html).” Many of the answers provided by 
Dr. Universe, however, do not mention 
particular faculty members or indicate who 
provided the actual answer to the question. 

There are several locations on the site 
that warn the user that it will take some time 
for Dr. Universe to answer the questions 
that are submitted and to ask users to be 
patient. From the Contents page, there is a 
link to a section called “Hey Kids”. This 
page expresses pleasant surprise over the 
number of questions that have been 
submitted but warns that it might take a 
little while to get to them all. Once a 
question has been submitted, there are two 
further cautions that there may be some 
delay in getting a response. Immediately 
after the question is submitted, the user 
receives a Thank You message for writing 
to Dr. Universe, and a request to please be 
patient as Dr. Universe is swamped. An 
acknowledgment is also sent to the e-mail 
address provided. This message also thanks 
the user for the question and says that it 
might take a while to get back to the user. 
There is no promise that the question will be 
answered within a certain time period. 

The Contents page of the web site and 
the section called “Why Dr. Universe?” 
indicate that answers are provided by WSU 
faculty. Some of the questions, such as the 
featured question, do identify the faculty 
member providing the information. Many 
responses do not mention the name or 
qualifications of the person answering the 
question. According to the site 
administrator. Dr. Universe staff consists of 
the site administrator and a part time 
graduate student. Questions are also given 
to students in a science teaching methods 
class as a final test. In most cases, the types 
of sources or names of specific sources used 
by any staff member are not provided. 

There is also no discussion of the question 
answering process, other than saying that 
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Dr. Universe will get answers from WSU 
faculty. 

There is no explicit statement about user 
privacy on the web site. The query form 
indicates that the submission of personal 
information is optional, and no personal 
information is displayed with the questions 
in the archive. The only personal 
information that is displayed with the 
questions is any information that was 
included in the query box when the question 
was submitted. There is also no statement 
of liability, responsibility or appropriate 
use. 



Ease of Use 

The service is very easy to find. There 
is a link from WSU's home page that leads 
directly to the Dr. Universe site. There is 
also a link from the WSU Research On-Line 
home page. The Contents page is the first 
page of substance that the user is brought to 
and it clearly displays most of the features 
of the site. The page is constructed using 
frames and the standard left side of the page 
contains links to: Today’s Question; 
Featured Questions; Submit Questions; 
E-mail the Dr.; The Dr.’s Home Page; WSU 
Home Page; and Research at WSU. Below 
these links are brief instructions for 
submitting a question online and via regular 
mail and a statement about the sponsors of 
Ask Dr. Universe. In the main portion of 
the contents page, there are large icons with 
text below them that lead to the main 
sections of the web site. These include: 
Featured Questions; Today’s Question; Find 
a Question; How can I Submit a Question; 
Links; and Dr. Universe Wallpaper. At the 
top of the page are two links, one to a 
section on “Who is Dr. Universe?” and one 
to a section called “Hey Kids”, which 
explains that it may take a while for a 
response. 

The Featured Question consists of a 
new question every other Monday that is 
answered at great lengths (current featured 
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question, 2 pages) by a researcher at WSU. 
At the bottom of this section is a link that 
says “Translate to Spanish, French, German 
or Italian”, and no further instructions. This 
link leads to the Alta Vista translation 
feature. A user could select the text of the 
question and paste it into the translator, but 
he would have to determine this himself. 

Today's Question contains a new 
question posted by a user daily. The answer 
is no longer than those of other questions, 
and there is no indication why a particular 
question is selected as Today's Question. 
Find a Question accesses the Dr. Universe 
Archive. How Can I Submit a Question 
brings the user to the query form. The user 
always sees the contents of the frame on the 
left side of the page, which details the other 
two options for submitting a question: by 
e-mail, or regular mail. The Links icon 
brings the user to a page containing Dr. 
Universe's favorite links. These include 
links to museums, other Q/A services, and 
sites specifically for children. Dr. Universe 
wallpaper contains a picture of Dr. Universe 
and instructions on how to set it as 
wallpaper. 

Administration 

Currently the service is free. It receives 
no specific funding and is run along with 
other departmental duties. The site 
administrator works for both News and 
Information and the Office of Research. In 
addition to running Dr. Universe, the site 
administrator is the research news 
coordinator and editor of the research 
magazine. The graduate assistant is a 
doctoral assistant who is funded by a 
consortium of schools, which are funded in 
turn by a State Department of Education 
grant. The site is anticipating major support 
from a corporate sponsor in the near future. 
The level of funding has not yet been 
determined. 
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Stafflng and Training 

The site administrator and the graduate 
assistant answer questions. As mentioned 
previously, questions are also given to 
students in a science teaching methods class 
as their final. They are given a brief 
presentation on the approach to answering 
questions and the site administrator reviews 
their answers before they are sent back to 
the users. The graduate assistant received 
no formal training, but the site administrator 
reviewed her answers for the first few 
weeks. Some questions, although it is 
difficult to determine how many, are 
answered by faculty and researchers at 
WSU. 

Answering Questions 

On the web site, there is no mention of 
the steps that are taken to answer a question 
and there are no guidelines for question 
negotiation. Some messages end with “did 
this answer your question?” or “is this 
helpful?” These responses indicate that 
users are free to resubmit questions or 
rephrase them to elicit a more appropriate 
response. The answers do not appear to use 
templates or canned language. The 
responses are very specific to the particular 
question or questions they are answering. 

There is generally not any 
personalization in the responses that appear 
on the web site, although random exceptions 
appear in the archive. The confirmation 
e-mail that users receive is addressed to the 
individual, in the form of “Hi Kristen!” 
Personal contacts through the web site are 
possible. Users can e-mail Dr. Universe 
directly and Dr. Universe will put the user 
in touch with a particutar faculty member or 
researcher, if they request it. Dr. Universe 
responded to a direct query in connection 
with this evaluation within 24 hours. 

The level of response varies from 
question to question. The answers range 
from a link to a web site to a sentence to a 
page or more. There is great variation in the 



amount of detail included in the responses. 
The response to a question about whether or 
not the Year 2000 problem is going to occur 
as many people fear was answered in five 
sentences with a promise that Dr. Universe 
would look into it further. A question on 
whether or not the year 2000 would be a 
leap year was answered in eight paragraphs 
and included a lengthy discussion and 
historical information provided by a 
historian at WSU. In general, the answers 
are informational rather than instructional. 
The responses are easy to understand and, 
according to the site administrator, the 
optional personal information is used to 
ensure that the level of the answer 
corresponds to the age or grade level of the 
user. This appears to be fairly accurate after 
inspecting questions in the archive. 

Inventory Control 

The service receives many more 
questions than they can answer. According 
to the site administrator, they receive an 
average of 600 questions per week and only 
respond to 15-20 percent of these. 

Questions are not rejected; they Just are 
unable to respond to them. There is no 
mention on the web site, and the site 
administrator did not indicate, how 
questions are prioritized and allocated. All 
questions are acknowledged by a 
confirmation e-mail. The user is cautioned 
that it may be a while before he receives a 
response, but he is not forewarned of the 
possibility of no response at all. 

Archiving 

There is a searchable archive of 
questions called Find A Question. The 
archive contains copies of questions 
submitted and the answers that are provided 
by the service. There is no identifying 
information, except what the user included 
in the box with the question. The user does 
not have the option of excluding his 
question from the archive. The archive is 
searchable by keyword, or the user can 
choose to browse through recently answered 
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questions. Once a question has been 
retrieved, the searcher can view a list of 
questions that contain the keyword(s) in the 
question. The searcher than must choose 
SELECT to view the response to a 
particular question. No further information, 
such as date, information about the user, etc. 
is provided. 

Hardware/Software 

The host machine for Dr. Universe is a 
dual 450Mhz Pentium II machine with 512 
MB RAM and 9GB hard drive. The backup 
drive is a 266Mhz Pentium II with 256 MB 
RAM and 3GB hard drive. Dr. Universe 
runs on Windows NT server and uses 
Microsoft SQL Server 6.5 as its database. 
Interactive web pages are built using 
Microsoft's Active Server Pages format. 
Other pages are standard HTML. A 
technical support staff member for the web 
site provided this information. 

Acknowledgment 

An automatically generated 
acknowledgment is sent immediately in 
response to all questions submitted to the 
service. The purpose of this 
acknowledgment may be to test the validity 
of the e-mail address, as in the AskEric 
service. The acknowledgment thanks the 
user for submitting a question and cautions 
about the response time lag. The 
acknowledgment also refers the user to the 
“Find a Question” section, so the user can 
check to see if his question had been 
submitted previously. Referring the user to 
this section before he submits his question 
may be more helpful. The user's question 
appears at the bottom of the 
acknowledgment. 

Evaluation 

No formal evaluation exists for the 
service. User feedback is not solicited. A 
user can e-mail Dr. Universe directly or 
send a letter expressing opinions on the 
service. There is no formal evaluation of 
responses to users' questions. After a 



several week period when the graduate 
student’s responses were reviewed, they are 
no longer monitored. The responses of the 
students in the teaching methods class are 
reviewed prior to sending them to the user. 

External Recognition 

The service has received recognition 
through awards and inclusion in newspaper 
articles. Specific awards that the service 
has received include; ProjectCool Site of 
the Day; Yahooligans Cool Site; USA 
Today Hot Site; New Scientist Site of the 
Day; Popular Science 50 Best of the Web; 
and Netscape's What's Cool Pick. The 
service has had major articles written about 
it in the Seattle Times and other area papers 
and has been mentioned in articles in the 
Wall Street Journal, USA Today, New 
Scientist, Los Angeles Times and The 
Baltimore Sun. Based on the quantity of 
questions currently received and the 
service’s inability to answer them, it does 
seem that the service needs publicity, 
although it could be helpful in leveraging 
corporate sponsorship. 

General Comments 

Overall, the service fulfills its mission 
to provide answers to questions that users 
ask. The service has some areas that could 
be improved with a larger budget and staff 
It would be helpful to have documentation 
on the web site that would explain the 
policies and procedures of the service, 
including the process involved in allocating 
and answering a question. Information 
about privacy, liability, responsibility and 
appropriate use would also be helpful. 
Documentation could include information 
about staff so users would know who was 
responding to their questions and what there 
qualifications are. 

There should be a standard format for 
the length of responses and the type of 
information included. Similar questions 
should receive comparable responses. 
Currently, some responses include extensive 




49 



43 



answers with historical information and 
referrals to other books or web sites while 
other questions are answered with only the 
minimum information needed to respond to 
the question. Responses should also cite the 
sources used and indicate who provided the 
information. The name and position of a 
faculty member or researcher is sometimes 
included, but most often, that is not the case. 

When submitting questions, it would be 
helpful if users had more information. Even 
simple instructions about forming their 
questions (use complete sentences, indicate 
what you already know) would be useful to 
both the user and the person answering the 
question. It would also be courteous to 
inform the users that there is a possibility 
that their questions may not be answered at 
all. The service could notify users that their 
questions cannot be answered at this time 
due to overwhelming demand and 
encourage them to resubmit their question at 
a later time. 

Another area that could be improved is 
the archive. Currently, users can only 
search by ke>^wd. They receive a list of 
questions that contain the keyword but have 
to select a particular question to view the 
answer. The questions contain no 
information other than what was entered 
into the query box. While it is not 
necessary, and probably would be unwise, 
to include a great deal of personal 



information about the user, the archive 
would be more useful if the query record 
also included at least the date and grade 
level or status of the user and allowed 
searching by these fields as well. 

As a whole, the general appearance and 
organization of the service is good. The 
icons are easy to locate and it is easy to 
move from one section of the site to 
another. The user can choose to submit a 
question from any page within the site. 

Also, despite its limitations, the archive is 
easily accessible and the user can search to 
see if questions on his topic have already 
been answered or view previously answered 
questions to get a sense of question format 
and content. One useful feature is the 
option to contact a faculty member directly 
concerning his area of research. There was 
no mention on the web site of how often this 
takes place, but it is an interesting idea. The 
service is entertaining and would be easy for 
even a young child to use. The answers to 
some of the questions, particularly the 
featured questions, are detailed and 
interesting and contain references to 
historical information and referrals to other 
sources. 

Sources 

Correspondence with Dr. Universe site 
administrator and technical support staff 
member via e-mail, June 1999. 
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Go Ask Alice!: Evaluative Analysis 

by Vera Welsh 



Mission, Objectives, Statement of 
Purpose 

Go Ask Alice! (http://www.goaskalice. 
columbia.edu/index.html) is a health-related 
question and answer (Q/A) Internet site 
created in 1993 by Columbia University's 
Health Education program. The mission of 
the service is to "provide factual, in-depth, 
straightforward and non-judgmental 
information to assist readers' decision 
making about their physical, sexual, 
emotional, and spiritual health 
(http://www.goaskalice.columbia.edu/about. 
html).” The following categories have been 
designated as topics that are addressed: 1 ) 
relationships, 2) sexuality, 3) sexual health, 
4) emotional health, 5) fitness & nutrition, 
6) alcohol, nicotine, other drugs, and 7) 
general health. 

The site boasts that it is "the first major 
Internet health Q&A site, originating in 
1993 (http://www.goaskalice.columbia.edu/ 
about.html).” 

Comments-. Although Go Ask Alice! is 
described as a Q/A service providing 
information related to all aspects of health, 
the service has become well known (and 
criticized by many) for its discussion of 
sexuality among teens and young adults. 
Whether the site overtly encourages such 
questions or whether it is a natural 
occurrence due to unsolicited interest is 
debatable. Ergonomically speaking, the 
layout of the web site may promote a higher 
volume of questions falling under the 
"sexuality" and "sexual health" questions as 
the other categories (emotional health, 
fitness, etc.) are displayed on the bottom 
row of icons. Sexuality and sexual health as 
two different categories while alcohol, 
nicotine and other drugs are treated as one 



category may lead some to believe the 
layout of the site influences the type of 
questions that are asked. 

Parameters of Service 

• Lengths of responses tend to vary from 
50 to 500 words on average. The only 
limitation placed on users is that they 
restrict the subject of their questions to 
the seven categories listed above. 

• All questions are read but only a 
"limited number" are answered for 
future posting. 

• There is no indication that repeat use is 
discouraged or prohibited. 

Comments: It is interesting to note that 
the service does not respond individually to 
the sender, but rather the sender will hope to 
see his/her question and answer posted on 
the site in the future. A sender will 
therefore not know why his/her question 
was not answered and may wonder: was it 
beyond the scope of the service? perhaps it 
was already posted in the archives? perhaps 
the answerer did not understand the 
question? 

This practice also increases regular 
readership of Go Ask Alice! by those who 
sent a question, as the sender will regularly 
check to see if his/her answer has been 
posted yet. Something to the effect of "if 
your question has not been posted within 
two weeks, assume it will not be" would be 
a welcomed notation on the site. The site 
boasts that it is "accessed more than 2.5 
million times a month;" a large percentage 
of those accessing are most likely looking 
for their question to be posted, not 
necessarily reading other questions and 
answers! 
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Identifying the Clients 

• Inquiries are anonymous. The 
addresses are electronically scrambled 
prior to reaching Alice's e-mail box. 
Initially the service was limited to 
Columbia University students only. 
However, the site became so successful 
that it was opened to the public. 

• The service does not request that any 
personal information is revealed from 
the sender (this includes those most 
basic - name, e-mail, age, grade level, 
etc). 

Comments'. Due to the personal nature 
of the types of questions that are asked and 
the age group that frequently uses the 
service (high school and young adult), the 
sender is granted both privacy and safety by 
not revealing personal information. 

This lack of identification is also 
valuable due to the personal nature of the 
questions that may be asked; a reluctant yet 
interested user may be more likely to use 
the service knowing that a guaranteed 
anonymity will be maintained. 

Due to the fact that the messages are not 
grouped according to age or education level, 
the service had to determine an overall level 
of sophistication to be used in answering 
questions. This is where the service may 
have become characterized as a site for 
teenagers and young adults without 
specifically stating that fact as the lowest 
common denominator was chosen. 
Generally, questions are answered at a level 
which a high school student can read and 
comprehend. Ask Alice! further adds that 
teachers may find some of the questions and 
answers useful in developing curriculum. 

Query Form 

• Although a form is provided, it is 
simply for the purpose of asking the 
question. Additional information is not 
requested, nor is guidance provided 
about how to phrase a question or what 



type of research has been previously 
carried out, as seen in some other Q/A 
services. 

• E-mail is the only option for sending a 
question. 

• Due to the popularity of the site, a 
number of newspapers and columns 
have expressed interest in using the 
questions and answers in their 
publications. Users are warned that 
these requests are not accepted in the 
question form. A separate link is 
provided for such inquiries. Comments 
and corrections are also not accepted as 
part of the question form. Users are 
linked to another form, which is 
prefaced by stating that a response is 
not guaranteed due to limited time and 
staff resources. 

A review of the site leads me to believe 
that the intention of Go Ask Alice! is not to 
serve as a personal Q/A service for an 
individual, but rather a means of soliciting 
questions that others may benefit from or be 
interested in reading as well. This suggests 
that the criteria for posting answers include 
the topic’s being interesting or useful to a 
large portion of the potential audience. 

Responsibility to the Clients 

• Because personal information is not 
solicited, user privacy is not addressed, 
except in terms of reminding users that 
personal information is not required. 

• Answerers are comprised of a team of 
Columbia University health educators 
and health care providers, as well as 
information and research specialists 
from health-related organizations 
worldwide. The names of persons and 
these organizations are not provided. 

• Users are informed that only a certain 
number of the over 1,000 questions sent 
each week will be answered and posted. 

(As noted earlier, this comment is 
extremely vague and leaves the user 
with little faith that his/her question will 
ever be one of the few posted). 
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• Prior to asking a question, the user is 
informed of a standard medical 
disclaimer, reiterating that all questions 
are not posted and therefore immediate 
responses should not be expected, the 
health information should not be 
considered specific medical advice, etc. 

o Users are encouraged to check the 
archives before asking a question. The 
archive contains over 1,500 questions 
and answers from the past. 

Ease of Use 

• The site is easy to use with an 
immediate link to the question form, 
information about the service, searching 
capability (and instructions on how to 
search) of the archives, as well as links 
to the specific categories with 
previously answered questions specific 
to that category listed. The site also 
uses includes advertising where its 
latest publications are displayed with a 
link to purchase/order information. 

• The e-mail address and other contact 
information are not provided for Go 
Ask Alice! Forms are followed by the 
submit button. This is the only direct 
means of communicating with the 
service. If one's browser does not 
support this type of arrangement, 
contact is not possible. There is a link 
provided to the homepage of 
Columbia's Health Department (sponsor 
of the service) with a web master's, 
address and contact information of 
faculty and administrators (names, 
phone number, address). However, it 
would most likely require several 
attempts to track down the appropriate 
source. 

Administration 

• The site is created and maintained by 
Columbia University's Health Education 
program. Information relating to 
specific funding is not discussed (e.g. 
grant, department budget, etc.) 

• Details of staff organization, influence 



in policy making, recruitment of 
answerers, etc., are not revealed. 

• Copyright is discussed but not as it 
relates to information used to answer a 
question but rather as it is used to 
protect the information produced by the 
service. Permission must be granted by 
Go Ask Alice! in order to publish or use 
the questions and answers by another 
individual or organization. 

• People (as opposed to an expert system) 
answer questions. It is unknown as to 
how questions are distributed, the order 
in which they are answered, etc. 

• The site offers no information on 
emergency or scheduled shutdowns. 
This would be useful information 
considering a university operates it. 

One wonders, for example, if it closes 
during academic recesses. 

Answering Questions 

As alluded to earlier, very little is 
known about the answering process based 
on information available on the site. Once a 
user sends a question, it may or may not be 
answered. Question negotiation/reference 
interview is not an option because the user's 
e-mail address is not requested. This results 
in an answerer reviewing a question and 
answering what he believes is the question. 
A user may view his/her question on the site 
and realize that the original question was 
not really answered. Questions are not 
referred. Several answers included links for 
additional information (e.g. a questions 
related to cancer provided a link to the 
National Cancer Society web site). Sources 
of answers are provided when used. 

Internet, print, and institutional sources are 
referred to and cited according to the Go 
Ask Alice! fact sheet. The majority of 
responses, however, seem to be based on 
expert knowledge. 

Inventory Control 

It is not known how questions are 
queued or ultimately selected. Criteria 
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and/or guidelines are not provided on the 
site. 

• Response time is not provided nor is 
there a guarantee of a response at all. 

• Every Friday, new questions and 
answers are posted. A link from the 
home page provides access, entitled 
"New Alice! Q & A's of the Week." 

• Questions that do not meet the 
guidelines are discarded. A reply is not 
given to the sender due to the fact that 
contact information for the sender is not 
solicited. 

Archiving 

• A searchable archive is available 
containing over 1,500 previously posted 
questions and answers. 

• Instructions and suggestions on how to 
search are provided. 

• The questions are arranged according to 
the seven topics outlined by the service, 
followed by a keyword representing the 
subject of the question in HTML (e.g. 
drinking water in NYC or bathing - how 
often?). Searching may be done by 
subject or title key word. It is unknown 
as to how long a certain item is 
archived, why certain questions are 
archived and not others, who manages 
the archive. It can be assumed that 
questions and answers deemed of 
greatest interest and still relevant are 
chosen to be archived. 

• The date in which the answer was 
originally posted is provided at the end 
of the question; this is a valuable piece 
of information. 

• There are no FAQ links as seen in other 
services. 

Comment: The archive is useful and the 
broad subject breakdowns provide some 
organization. However, efficient searching 
is somewhat challenging. Unless a user 
uses the exact wording used in the 
question/answer or can deduce what the 
question is about based on a word or phrase 
used to represent the questions, he will 



probably browse through several archives 
before retrieving what he/she really wants. 

Quality Control 

• The following statement is made 
regarding quality: "Responses undergo a 
standardized review process to insure 
high quality and accuracy. 

Comment: Again, it is not known what 
the review process entails, but it is 
reassuring to know the service recognizes 
the need and has a system in place. 

Hardware/Software 

• The hardware/software used by the 
system in unknown 

Acknowledgment 
Question Negotiation 

Neither of these is possible because the 
user submits no personal information. 

Response Guidelines 

• The response falls under the category of 
"free writing"; forms, templates, etc. are 
not used. 

• All answers are signed "Alice" as 
opposed to providing any personal 
information. 

• As emphasized in the Go Ask Alice! 
Fact sheet, the service strives to provide 
objective responses. In the case of 
sexuality for example, this is critical. 
The service does not condone or 
advocate a certain group of values or 
morals. 

• Answers to questions are in the form of 
answers, as opposed to any type of 
instruction or steps to find an answer. 
When additional information is 
provided, included in the answers are 
the citations or web links to the sites. 

• It is assumed that ineligible and 
inappropriate questions are discarded, 
along with those not answered due to 
high volume due to the fact that contact 
information is not solicited from the 
sender. 



• Generally, in answering a question, it 
appears that anyone with at least a high 
school level of reading and 
comprehension could easily understand 
the response. Although the site states 
its audience includes high school, 
college aged, parents, professionals, 
etc., the answers are presented at the 
lowest level of audience. 

Evaluation 

Information about evaluation of the 
service is unknown. The service has been in 
place for over six years and attracts a high 
volume of users, both for asking questions 
and for browsing. The site reveals numbers 
on the specific volume of questions asked 
and the number of time the site is accessed, 
so statistics are being maintained. 

External Recognition 

The Go Ask Alice! site provides a list of 
over twenty well-known organizations and 
publishers that have praised the site, 
including The New York Times and Harvard 
Health Letter. 



KidsConnect: Evaluative Analysis 

by Kathleen Feeney Chappell 



Mission, Objectives, Statement of 
Purpose 

KidsConnect, located at http://www. 
ala.org/ICONN/AskKC.html, is a free 
online question and answer (Q/A) and 
referral service meant to help K-12 students 
find information on the Internet and 
elsewhere to use for school projects. The 
name makes more sense after a user sees the 
service: connecting K-12 students to 
resources. But, if all that is known is the 
name and that it is a Q/A service, the name 
is not very descriptive. The KidsConnect 
home page indicates that KidsConnect is 
meant to help K-12 students find Internet 
and print resources to use for assignments. 

If the user follows the link to the 
information for teachers, parents and school 
librarians, that page, too, says the service is 
intended to help students access and use the 
information available on the Internet 
effectively and efficiently. The user will get 
a better idea of the service’s mission if he or 
she reads all of the information on the site, 
but most users will probably only look at 
the first page. 

Parameters of Service 

The service is aimed at K-12 students, 
and the answers consist of lists of Internet 
sites and print resources relevant to the 
user's query. It is meant to help students 
with their assignments for school. There are 
no limits on how many questions can be 
asked, which types, length, etc. From the 
web page, the user cannot tell if 
KidsConnect rejects questions. It seems as 
though all questions submitted will be 
answered. The only limits on the service 
appear to be that, since the service has a 
2-school-day time limit, users cannot ask 
for information they need to use right away. 



Comments: The idea of providing the 
students with only lists of sources to 
consider is an interesting one. It almost 
completely removes the issue of whether or 
not the service provides factual or correct 
information (although the links and citations 
provided have to be accurate, of course). It 
also serves to educate the students, teaching 
them how to find resources on the Internet. 
Since the users are not provided directly 
with the information they are looking for, it 
seems like there is a better chance that they 
will learn something beyond the factual 
information they have to find for their 
assignment. 

Identifying the Clients 

KidsConnect clients are K-12 students 
who need help finding Internet and print 
resources for school assignments. 

Query Form 

The form asks the student to provide: 

1. First name. 

2. E-mail address. 

3. Grade. 

4. Topic. 

5. Resources previously consulted. 

6. Kind of assignment (with examples 
"science fair project? five page report? 
chart with information? something 
else?"). 

7. Question. 

The user can also just send an e-mail, and 
all the same guidelines are suggested. 

Comments: I like that the question 
components are included in two different 
ways; it seems that there is a better chance 
that the users will include everything this 
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way. It is also good for the users who do 
not use the form, since they will know what 
to include in their question. Asking only for 
the first name allows the responses to be 
personalized but removes some privacy 
concerns. 

Responsibility to the Clients 

KidsConnect indicates it will provide 
the client with suggestions of good web 
sites and other resources to look at for help 
with his or her assignment. The web site 
states that answers will usually be returned 
within 2 school days, that the answers will 
be provided by volunteer school librarians, 
and that the types of sources users will be 
referred to include web sites, encyclopedias, 
almanacs, books, magazines. All of this 
information is provided on the first page, 
before the user gets to the query form. 

The KidsConnect service does provide 
information about how the questions are 
answered, but the user has to look for it. 

The first page of the web site, the one aimed 
at kids, only tells who will answer the 
question; the page for parents is not much 
more informative; the page for school 
librarians who want to volunteer for the 
service is much more explicit; explaining 
how the volunteers are organized into teams 
and that team leaders distribute the 
questions to the volunteers. 

Comments: I like that there are separate 
pages for the kids and the adults to look at, 
since putting detailed information on the 
page aimed at the kids would probably 
make it too long and it would just get 
ignored anyway. By sending the adults to a 
different page, the people who are more 
interested in the information can learn how 
the service works without having the 
information clutter up the page with the 
query form. It is good that it exists online, 
though, and kids who were really interested 
in how the service works can look at it, too. 



The service does not address user 
privacy, and there are not really any 
statements of liability or responsibility. 
There is a statement on the page warning 
users that not all information on the Internet 
is reliable and that they should check with 
their own school librarians for help. The 
volunteers are also instructed to encourage 
the students to check with their school 
librarians in every reply they send. There 
does not seem to be much liability involved 
with this service, since it does not actually 
providing the user with answers, just 
guidance about where to find answers. 

Ease of Use 

The query form is at the bottom of the 
first page, after the introductory 
information. The user can also click on a 
link near the top of the page to take them 
directly to the form or the e-mail address. 
This means that even someone who uses the 
query form can see the e-mail address for 
the service. The only choice for getting a 
response seems to be e-mail, because the 
form does not ask for phone numbers or 
give the user other response options. 

Comments: Setting up the page in this 
way, with the form at the bottom but with 
an option to go straight there, seems like a 
good idea to me, since it means that repeat 
users can skip over the basic information, 
which they have already seen. The 
information at the beginning about what to 
include in the question is reproduced on the 
form, so that does not get lost, and, if a 
student has used the service already, he or 
she should know what to expect in terms of 
an answer. 

The page with the query form 
recommends that the user look at the site's 
Frequently Asked Questions (FAQ) and the 
KidsConnect Favorite Web Sites resources 
if he needs help fast, but it does not instruct 
him to look there before asking a question 
or instead of asking one. 
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Comments: This seems like a more 
user-friendly format than the services that 
say, "You MUST check to see if we've 
already answered your question before you 
ask it." It makes it seem as though 
KidsConnect is really there to answer 
questions, not scare away users. Maybe this 
service gets less use than some others do, 
though. 

Administration 

KidsConnect is funded by Microsoft 
(the page says it is "underwritten" by 
Microsoft, which makes it sound more 
pleasant). There are several other 
organizations involved with the project, 
which is a part of ICONnect, an initiative of 
the American Association of School 
Librarians (AASL), a committee of the 
American Library Association (ALA). 
Syracuse University and AskERIC, which is 
based at Syracuse, provide design, 
development, and technical services for 
KidsConnect (Microsoft ..., p. 229). In an 
interview with the Virtual Reference Desk, 
Blythe Bennett, the coordinator of the 
service, explains that the service was 
originally conceived of as a kids' version of 
AskERIC (http://www.vrd.org/AskA/ 
Spotlight/spotlight_KC.html). The user is 
unable to tell from the web site how these 
groups work together and who has how 
much say in what. It does seem that AASL 
is the major player though, since all of the 
volunteers are members of this group. 

Staff is organized into teams of 8-10 
volunteers who work during the same month 
and (it seems there is a month-on, month-off 
system in place). The librarians are grouped 
by level: elementary school librarians 
answer questions from elementary school 
students, and so on. Each team has a team 
leader who serves a mentor to the other 
team members, distributes the questions, 
does any necessary trouble-shooting, and 
monitors the quality of the responses written 
by the other team members. Each team has 
a partner team, which volunteers in their 



off-month. The larger structure of the 
organization is unclear from the web site, 
though. It is not apparent from the web site, 
for example, how the service fits into the 
AASL structure . 

Similarly, it is unclear whether 
KidsConnect has general emergency 
procedures or how much the staff of the 
service is involved in making policy 
decisions like these. 

Staffing and Training 

Volunteers answer the questions 
submitted to KidsConnect. The volunteers 
are all K-12 school librarians who are 
members of AASL; as of May 1999, 
KidsConnect had 230 volunteers. There do 
not seem to be any qualifications required 
beyond these professional ones, and 
volunteers do not even have to have a lot of 
experience using the Internet. Librarians 
can volunteer for the service through the 
web site. Volunteers are also recruited 
through postings to school library listservs 
and in professional publications, as well as 
through presentations at conferences 
(Bennett, p. 126). It is unclear how long 
most volunteers remain involved with the 
project. 

Comments: KidsConnect seems to 
serve as a public relations tool for the 
AASL members. Each response encourages 
the student to consult with his own school 
librarians about the questions he submits. 1 
also think it is good advice though. 

The volunteers must go through on-line 
training, which, according to AskA Starter 
Kit, involves reading an online manual 
describing the service, looking at 
information on how to search the web and 
answering practice questions. The new 
volunteers answer several questions that 
have been previously answered by the 
service; their responses are reviewed by the 
service coordinator or another trainer and 
returned with feedback. This review 
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process continues until the trainers are 
satisfied with the quality of the responses 
and the volunteers feel comfortable enough 
to answer questions on their own. After this 
point, the librarians join a team of question 
answerers. Volunteers who do a good Job 
answering questions have the opportunity to 
become team leaders, who check over the 
responses of team members (Bennett, pp. 
126 - 127 ). 

Answering Questions 

The questions are sent to the 
KidsConnect main address and are routed to 
volunteers by the team leaders. After each 
question is received, the server sends an 
automatic response to the sender, stating 
that the question has been received. The 
volunteer librarians answer the questions 
and send the responses back to the student 
(or to the team leader, who checks quality, 
then to the student? This is unclear). There 
do not seem to be question negotiation 
guidelines, templates, or much 
personalization provided in the answers. 

The librarian signs the response with his or 
her initials, based on responses in the FAQs. 
Personal contacts do not seem to be 
encouraged. 

The KidsConnect web site does not give 
very much information about how questions 
are to be answered, other than saying that 
volunteer librarians should tell users to 
consult with their school librarians. This 
could be seen as a method of referring the 
question, but there are always resources 
provided in the answer, too. It does not 
seem that the librarians refer the questions 
to anyone outside KidsConnect. 

Inventory Control 

When a question is sent to KidsConnect, 
it goes to the main account, and the team 
leaders distribute the questions to the 
volunteers. From the web site, it is unclear 
how this is done, if certain librarians answer 
only questions on certain topics or if 
questions are prioritized on the basis of how 



easy or hard they will be to answer, or if 
questions are ever rejected completely. 
According the YRD interview with Bennett, 
inappropriate questions (defined by her as 
questions that are not from K-12 students or 
parents, teachers or librarians dealing with 
this age group, and questions asking for 
advice instead of resources) are deleted 
from the service's inbox, and the questioner 
is notified that the question is out of scope. 
Presumably, time is a factor in determining 
which questions to answer first, since the 
service aims to answer all questions within 
two school days. There are no guidelines on 
the web site that instruct volunteers how 
long to spend attempting to answer an 
individual question. 

Coping with Unexpected Demand 

It is unclear whether the system 
experiences scheduled down times and, if 
so, if they are communicated to the users. It 
is also unclear what would happen to the 
system if there were more questions than it 
could handle if the system crashed. 

Archiving 

There is no archive of KidsConnect 
questions available on the web site, 
although there is a list of frequently asked 
questions. The FAQ is indexed by subject 
area and by topic. It is not searchable. 

There is no information on the web site 
about how questions and answers are chosen 
for the FAQ or how it is related to an 
archive of questions, if one exists. 

Comments: The list of subject headings 
for the FAQ is broad, while the list of topics 
is very specific, including "Biomes," 

"Martin Luther King, Jr.," and "Toothpick 
(and Other) Bridges." No context is given 
for the questions that are included, which is 
too bad. Some of the questions are broad 
enough that the resources given for them 
could be useful for other questions, but the 
web page does not tell users this. Users 
may be able to extrapolate from the Martin 
Luther King question the fact that they can 
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search for information on another person by 
doing a similar AltaVista search, but it 
would be helpful if the page mentioned this. 

Quality Control 

The team leaders monitor quality of the 
answers, but it is not clear from the web site 
what standards they use, nor is it clear what 
happens to an answer that is considered 
inadequate. 

Hardware/Software 

The web site does not give any 
information about the hardware or software 
that KidsConnect uses. In the VRD 
interview, Bennett states that KidsConnect 
currently uses the PINE e-mail system for 
routing and answering questions. However, 
the service plans to switch over to Eudora 
in the future. 

Acknowledgment 

An automatic message is sent by the 
server upon receipt of a question, informing 
the user that the question has been received. 
It also reminds users to cite the sources 
they decide to use for their assignment. 

Question Negotiation 

The web site does not mention anything 
about question negotiation. It seems 
unlikely that question negotiation would 
happen often since the service is striving to 
meet the 2-school-day time limit. 

Response Guidelines 

Based on the messages that are in the 
FAQ, the responses are free form. There 
seem to be no limitations on the length of 
the answers provided; in fact, some of them 
are quite long, providing links to many web 
sites and several citations for print sources 
as well. Since the volunteer librarians are 
not actually providing factual answers, they 
do not have to document sources. However, 
if the librarian used a web search engine 
like AltaVista to come up with web sites on 
the topic of the question, the answer 
includes the search terms the librarian used. 



Including search terms is a good tactic 
because it enables the student to redo the 
search at a later date if desired, and it may 
help him learn about online searching, as 
well. 

Several things about response 
guidelines are not explained on the web site. 
It is not clear whether there are policies for 
dealing with questions that are confusing or 
outside the scope of the service. 

Presumably the volunteer librarians will use 
the grade level specified by the student on 
the question form to tailor their answers, but 
it is unclear how this is done or if any other 
factors influence the nature of the response. 
It is also unclear whether there is any 
policy about the sources to be used; the web 
site includes a list of "Favorite Web Sites," 
but it is not clear from the site if the 
volunteers are encouraged to use them in 
their answers. It was not evident from the 
site if KidsConnect has a private archive, so 
it is not possible to say if old answers are 
recycled for later use. The web site also 
does not mention whether the questions 
received are prioritized in any way or if they 
are just answered as they come in. 

Answers are returned by direct e-mail to 
the student who asked the question within 
two school days. It is not apparent if the 
student can request clarification of the 
answer he or she was given or if further 
dialogue with the service is encouraged. 

Evaluation 

The only information the KidsConnect 
web site gives about evaluation is that the 
team leaders monitor the quality of the 
answers that the volunteer librarians 
provide, but it is not clear how this is done. 
It is not clear if the service maintains any 
statistics on use or customer satisfaction. 
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External Recognition 

The site does not seem to have won any 
awards. 
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LawGuru.com BBS: Evaluative Analysis 

by Bryan Faganl 



The name of the site is LawGuru.com, 
although the name of the actual "Ask a" 
service is never clearly stated; Pitsco’s Ask 
an Expert lists it as "Ask LawGuru.com," 
and the site itself calls it "LawGuru.com 
BBS" most frequently. It also uses generic 
language such as "Legal Questions" and 
"Find Free Answers to your Legal 
Questions," however. ’’LawGuru” does 
seem to get across the message of the site, 
that it will answer user questions, and 
**free** is emphasized so much it is almost 
impossible to miss. The URL for the main 
site is http://www.lawguru.com/, and the 
actual question form is at http://www. 
Iawguru.com/cgi/bbs2/user/submit.cgi. The 
site’s goal is to "try to help people find free 
answers to some of their legal questions." 
This is relatively obvious once the 
question part of the site is reached, 
although it is not well stated from the 
front page of the site. 

The service answers both simple and 
complex legal questions, although the 
accuracy of the more complex cases is 
obviously affected by the amount of 
information the user puts into the question. 
The service rejects few questions on the 
basis of relevancy; the rest are either 
answered by the Los Angeles law firm that 
operates the site, Eslamboly & Barlavi, or 
posted on the site. Eslamboly & Barlavi 
answers all the questions that fall within the 
specialties of the attorneys at their firm, but 
the site does acknowledge some questions 
will not be answered even though they are 
posted because no lawyers affiliated with 
the service practice in the specified state or 
area of law. Users are asked to keep their 



1 The author’s comments are in bold. 



questions to one short message per form. 

The intended users for the system are 
users of the web who have a limited 
knowledge of the law. The only common 
characteristic is access to a computer. The 
BBS covers such a wide variety of types 
of law that it does not seemed to be aimed 
for a specific legal niche: categories 
include areas such as criminal, 
environmental, bankruptcy, divorce, and 
aviation law. LawGuru.com seems to try 
its best to make the service inviting to all 
people who have questions about the law. 
On its question form, there are even 
blanks for foreign locations. 

Both forms and e-mail communication 
are acceptable ways to send questions, 
although it is quite clear that the form is 
preferred. Name, state (for purposes of 
determining which state's laws are 
applicable; uses a pull-down menu), zip 
code, e-mail address, heading, area of law 
(uses a pull-down menu), and the question 
are all required blanks on the form, each 
marked by a small red square. Heading and 
area of law seem to combine into a 
’’subject” area. Users who opt to use e- 
mail (lawguru@lawguru.com) are given 
little if any guidance. Presumably, the 
form should be used to guide the form of 
the e-mail. No guidance is given on how to 
ask a question, although users can search for 
questions that have been previously asked to 
determine which received the most 
complete and lengthy responses. Although 
this seems like a shot in the dark, 
studying questions in the same area as the 
user’s question is a good predictor, since 
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the same attorneys will be answering the 
questions. However, users may not think 
of this approach, and it certainly is not 
spelled out. 

In addition, users are required to type "I 
agree" to a release that allows Eslamboly & 
Barlavi to post the user's question. 

Although this seems unusual, it is logical 
that a law site would want to cover itself 
and would realize that clicking radio 
buttons does not convey the level of 
comprehension that having to read 
through a disclaimer to And out what to 
type in a blank does. Blanks for firm, street 
address, city, country, non-U. S. state, day 
and evening telephone numbers, and fax 
numbers are also present, and a pull-down 
menu allows users to choose a different 
state if the legal question they are asking 
about does not correspond to the state in 
which they reside. In addition, a section 
marked "To be able to serve our users 
better, we would also be interested in 
knowing" solicits information about users' 
age and occupation, and it asks how users 
heard about the service. No mention is 
made anywhere on the site about how this 
information will be used. It also allows 
users to specify whether users would like to 
be automatically informed when the web 
site is updated. 

The question-answering service is one 
screen down from the top and not given 
much fanfare, but it takes only one link to 
reach it. The top of the page is dedicated to 
a banner advertisement, a "What's New" 
statement, and legal search engines. The 
"Legal Questions" section actually takes up 
less space than any other feature on the 
front page of the site. Besides the main link 
from the front page of LawGuru. com, there 
is also a "Questions and Answers" link on 
the same page; it is not at all obvious that 
this link leads to the question-answering 
section, and the title of the link is rather 
generic rather than evoking any sort of 
an "Ask a" service. Other than the in 



obvious "Questions and Answers," 
navigating the site is simple and self- 
explanatory. 

The service makes it clear users will get 
a response to their questions only if they 
meet the legal expertise of the lawyers 
associated with LawGuru.com. The service 
goes to pains to remind users that answering 
questions does not in any way establish a 
lawyer/client relationship. As soon as a 
question is received, the system 
automatically sends the user an e-mail 
stating that, when the question is posted to 
the BBS, which can take up to two weeks, 
the user will receive another confirmatory e- 
mail. When I asked a question to the 
service, it took only one day for the 
question to be posted. Obviously, 
turnaround time can vary greatly. 
Eslamboly & Barlavi takes the first shot at 
the question. If it is outside the firm's 
purview, which almost all questions dealing 
with state law other than California's will 
almost certainly be, the question will be 
posted to the BBS, and lawyers whose 
profiles are matched to the question will be 
notified by a computer program. The 
lawyers use their own expertise to answer 
the questions. When a response is posted, 
the user will get another e-mail. The site's 
procedures for distributing and answering 
questions are repeated throughout the site, 
appearing on the help page, the main 
question service page, and on the page with 
the question form, although it does not 
mention what resources the lawyers will 
use. The redundancy of these procedures 
gives even the most careless users a 
chance to understand what is happening. 

The site's commitment to privacy is 
detailed on the page with the question form. 
Personal information, which includes name, 
e-mail address, phone number, etc., is 
obscured from all members of the BBS, 
including member attorneys. Only the 
administrators and attorneys from 
Eslamboly & Barlavi can access that 
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information. In addition, privacy is 
addressed on the disclaimer page, reminding 
users that transmissions on the Internet are 
not always as private as intended, and users 
are transmitting the information at their own 
risk. The disclaimer states that the service is 
"provided for... information only and should 
not be relied on as legal advice." 
Furthermore, the disclaimer warns users not 
to rely on information gleaned from the site 
without consulting a practicing lawyer. 

Users are warned that no client-attorney 
relationship is established between the users 
and either Eslamboly & Barlavi or the 
lawyers who answer their questions. The 
accuracy of answers is not guaranteed, and 
none of the information on the site should 
be taken as an endorsement of a product or 
service (in the case of banner ads or law 
firms who answer the questions). 
Unsurprisingly, a law "Ask a" site tries to 
cover its legal bases, although the 
disclaimers are short, to the point, and 
easily comprehensible. 

The main page of the service sends the 
user to "Internet Law Library" (http://www. 
lawguru.com/ilawlib/index.html) and the 
"Legal Research Meta search page" 
(http://www.lawguru.com/search/lawsearch. 
html). The service itself directs the user to 
previously asked questions as much as it 
directs them to ask their own questions; 
when the option is given, the archive of 
previous questions is mentioned first. The 
destination e-mail address is not explicitly 
listed on the form, but it is not difficult to 
figure out the information will go to a 
central distribution address at 
LawGuru.com. The information is posted on 
the BBS only, although confirmatory e- 
mails are sent when the question is received, 
posted, and answered. 

The service is operated by a commercial 
law firm, and the firm also solicits 
commercial advertisements (banner ads for 
mortgage and finance companies were 
easily apparent when I visited the site; 



other businesses may also advertise). No 
fees are charged, and "free" access was 
stressed throughout the site. The advertising 
and vested interests of one law firm does 
have a chance to influence objectivity, 
although the disclaimer states that ads or 
other information should not be seen as 
endorsements. There is a potential, not 
adequately addressed in disclaimers, for 
Eslamboly & Barlavi to use the service to 
tantalize users (e.g., overestimating the 
chance of a successful court action without 
completely revealing the law behind it and 
telling the user they have handled many 
such cases). I did not see any evidence 
that Eslamboly & Barlavi did this, but 
other law firms seemed to be more 
interested in attracting clients than 
answering questions. Also, one of the first 
points mentioned on the volunteer 
application page is that it can be used to 
attract business. Law firms may see this 
not as a service but as a way to draw new 
clients by any means. 

"Staff," as in those who answer 
questions for the service, are not arranged in 
any manner specified by Eslamboly & 
Barlavi. An expert system matches the state 
and subject area of a question to the profile 
filed by a lawyer, and every lawyer whose 
profile matches the specified state/subject 
criteria is notified when a question is posted 
to the BBS. None of these attorneys is 
required to answer, however. 

The ethical/privacy statements are 
comprehensible, readily apparent, and easily 
accessible. The site does not, however, 
address copyright, fair-use, or selective 
dissemination of information questions. As 
stated above, Eslamboly & Barlavi makes 
no guarantee of the accuracy of the 
information on the site. No general 
emergency procedures are listed, and there 
is no mention of staff involvement in policy 
development. The latter is not 
surprising, given the extremely voluntary 
nature of the service. 
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Volunteer lawyers in more than 350 
firms staff the site across the country. A law 
degree seems to be the only qualification 
needed, although malpractice insurance 
information is required in the attorney 
volunteer application. (Name, firm, number 
of attorneys in the firm, mailing address, 
states in which the firm practices, a phone 
number, an e-mail address, areas of practice. 
Bar number, length of service to the Bar, 
and whether the lawyer is an insurance 
defense attorney are required. Other phone 
numbers, fax number, other e-mail 
addresses, and web page are asked for but 
not required. Attorneys are also required to 
choose their own login and password for the 
BBS, and they can also specify what 
appears in their public profile on the site, 
including a blurb about themselves.) The 
service does not recruit, as such; lawyers 
find out about the service through the web 
site, word of mouth, mailing lists, news 
groups, etc. Apparently, all who apply are 
accepted. Since more than one lawyer can 
reply to a question, having many lawyers 
in a subject area can be an asset rather 
than liability since questions are not 
divided between a pool of experts. A 
variety of expertise and experience is 
encouraged. 

Training is minimal, restricted to a few 
general guidelines for lawyers on the 
attorney application page. This lack of 
training, along with a lack of quality 
control and other reasons, lead to a 
considerable variance in the quality of 
responses. There are no seniority benefits I 
increased responsibilities for those who 
have been witlrLawGuru.com the longest. 
No information about the expected time 
commitment was available on the site. 

When a question is received on 
LawGuru.com BBS, a confirmatory e-mail 
is sent to the user. Then, an attorney at 
Eslamboly & Barlavi reviews the question; 
the question will either be answered by the 



firm, in which case the answer will be e- 
mailed directly to the user, or it will be 
posted to the BBS. If it is posted to the BBS, 
the user will get another e-mail to update 
him/her on the status of the question, and 
attorneys whose profiles are matched to the 
criteria of the question by the site's software 
will receive an e-mail apprizing them that 
the question has been posted. Lawyers will 
review the question and respond; multiple 
unique responses can be attached to each 
question. Each time an answer is posted to 
the question, the user who posted the 
question will be notified via e-mail. Like the 
Mad Scientist site, the question and 
answer(s) cannot be viewed at the same 
time; however, LawGuru.com differs in that 
users click on the question first, and links to 
all answers that have been posted are listed. 
Each answer is given a separate link and 
page. Although this is more difficult to 
navigate, it does help separate the 
responses, which competing laiwyers will 
no doubt appreciate. 

Volunteers cannot negotiate questions 
with users because all personal information 
is withheld, and no one is under any 
obligation to answer a question. In any 
event, LawGuru.com and its volunteers 
are probably eager to avoid anything that 
resembles a client/attorney relationship. 
Templates existed in version 1 .0 of the site's 
software, but version 2.0 does not support 
them yet. Personalization of responses is 
vital; it is the only way users can 
communicate with lawyers who replied to 
their question if they want to consult. Most 
referrals given on the site are very general, 
as in "Consult with an attorney," although 
those who ask questions that are not covered 
by LawGuru or contain insufficient 
information to be answered are directed to 
"other possible sources of information," 
according to Bahman Eslamboly. Sources to 
answers, in the form of legal codes and 
statutes, are given occasionally, but there 
seems to be no hard-and-fast rule about 
citations. No substantive guidelines about 




65 



59 



the nature or time of the response exist on 
the site. 

After questions are either answered or 
posted on the BBS by Eslamboly & Barlavi, 
they no longer receive attention from any 
specific person. Because of the BBS system, 
no question receives priority over another: 
even those that have not been answered are 
not given preference over those that have 
already have been. This does not help 
users, but at least it ensures responses 
will be given by someone who actually 
wants to answer. Certain areas of law 
that are more profitable may receive 
more attention than less profitable ones, 
however. All communications are 
immediately acknowledged, usually in an 
automatic e-mail. There are no guides on 
how much time should be spent on 
questions or on what is done with 
inappropriate questions. The site also does 
not mention anything about scheduled 
downtimes or system capabilities. Even 
though the site does not tell users about how 
interruptions of service will be conveyed, 
users have the option, when sending 
feedback or a question, to specify they want 
to be informed about changes to the site. 

The site has no archive per se. Each 
question is posted to a BBS and stays there 
perpetually, though, so users can see the 
questions that have been asked. The BBS is 
accessible by anyone and includes all 
questions asked throughout the BBS's 
history, so it is not selective in the slightest. 
The BBS contains over 3,500 questions and 
answers. The purpose of the archive is the 
same as the question-answering service: to 
help people "try to find answers to legal 
questions." No personal information is 
attached to any of the questions in the BBS. 
Users are not allowed to specify that they do 
not want their question posted to the BBS, 
although given the system in use at 
LawGuru.com, not wanting the question 
posted would be nearly tantamount to not 
wanting the question answered. The BBS 



is easily searchable by state and subject area 
or by keywords. There is no FAQ, although 
there is a "Help" section that seems to 
answer basic questions about how the 
question-answering service works. 

Centrally enforced quality control is 
absent from the site, although employees of 
Eslamboly & Barlavi "sometimes" review 
the answers. However, since more than one 
lawyer is allowed to answer each 
question, answers can be compared to one 
another for some questions. Although 
lawyers may try to curry favor with users 
by providing the information users wish 
to see, this approach can encourage 
competition to provide the best answers. 
The site does not mention what becomes of 
answers or volunteers that are deemed to be 
of "inadequate" quality. 

All questions are immediately 
acknowledged by computer-generated e- 
mail that contains no human input. The e- 
mail contains the date the question was 
received and the title of the question. User 
are informed that "BBS administrators have 
been alerted that a new question has been 
received" and that they will receive another 
e-mail informing them of whether the 
question has been accepted or rejected. The 
message also mentions it may "take a 
couple of weeks before our question is 
reviewed prior to posting." 

Responses to the questions are 
generated from free text. Lawyers 
responding to questions have the freedom to 
respond in any way that they think will 
answer users' questions. Few constraints are 
placed upon the responses at all, as lawyers 
are allowed to use their own judgement on 
what to cite (if anything), how much 
judgement to include, and how long the 
answer should be. However, the emphasis is 
on getting answers to users of the service, so 
answers contain more actual information 
rather than instruction. This is natural 
enough, considering the fear most people 
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hold "legalese" in and how monolithic 
and impenetrable laws and rulings can 
seem to users. Occasional law citations are 
included as well. There are no ineligible 
questioners, but 

inappropriate/incomprehensible questions 
are referred to "other possible sources of 
information." 

Answers on LawGuru.com seem to 
reflect knowledge that the most of its users 
are laymen and will not understand jargon. 
No restrictions are placed on what 
constitutes a relevant source and when one 
should be used to answer questions. 

Previous answers, including, occasionally, 
previous answers for the same question, are 
available for volunteers who need guidance 
for the structure of their own answers. Most 
answers (those not answered by Eslamboly 
& Barlavi attorneys) are posted on the 
bulletin board, and the users get e-mail 
informing them that an answer for the 
question has been posted. Users whose 
questions fall into the area of expertise of a 
lawyer for Eslamboly & Barlavi will get a 
direct e-mail answer to their questions, 
however. No direct clarification can be 
requested through the service, and clients 
can either follow-up by contacting the 
lawyer who answered their question or by 
asking LawGuru.com another question. No 
policies on acceptable response times or 
priority exist. Since it is difficult to attach 
non-textual materials to a BBS answer, the 
possibility is not covered in policies, nor is 
there a policy on referring users to non-text 
resources. 

LawGuru has no formal evaluation 
program in place, although they do collect 
information that is not used in answering the 
question from each user. 

The site has won several awards, 
including the World Wide Legal 
Information Association's Five-Star Award, 
the Starting Point (http://www.stpt.com/) 
Choice Award, and oddly enough, the Adult 



Webmasters' Excellent Resource Award. 
Make what you will of that last one; the 
"Adult" means exactly what you think it 
does. It has also been listed as an 
"exceptional site" by Pehrson Web Group 
Mentoring. A few articles in the general 
legal press mention LawGuru.com, but most 
of them concentrate on the legal search 
engines on the main page. Only one article 
mentioned the site's "ask a" service, and 
even that was only in passing. 

Overall, the site is not governed much 
at all. Eslamboly & Barlavi seems to 
provide the forum for the service and 
skim the questions it wants to answer off 
the top more than it seems to administer 
the site. In some ways, the freedom works 
well; attorneys who are the most 
interested in a question are the ones who 
will answer it, and if there is any follow- 
up, those attorneys will be in the best 
position to work with the user. However, 
this freedom also allows user questions to 
go unanswered, and questions can 
languish in the BBS for months without a 
response. Answers can also show a great 
range of quality, but this is not 
uncommon, even with "Ask a" sites that 
have a great deal of traffic and high 
standards of quality control, such as the 
Mad Scientist. However, Eslamboly and 
Barlavi's virtually hands-off stance on 
answer quality will probably only 
exacerbate the situation. 
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The Medical Edge’s Answer Doc: Evaluative Analysis 

by Rick Whitaker^ 



The Medical Edge (http://www. 
mededge.com/ is a profit-making 
organization, which provides a range of 
medical information online. Its electronic 
question and answer (Q/A) service is 
AnswerDoc (http://www.mededge. 
com/whoask.htm). This service offers 
several levels of electronic question/answer 
services, including both free and fee 
services. 

Mission, Objectives, Statement of 
Purpose 

The Medical Edge's mission statement 
is: 

The Medical Edge is an interactive 
online community resource 
designed to provide medical 
information, products and services 
to the health care consumer. Our 
goal is to become "The one stop 
shop for all your health care needs". 

We plan to appeal to the veteran 
Internet user and to draw the 
attention of those who have not 
considered the relevance of Internet 
technology in their lives. Aiming 
to bridge the gap between the digital 
revolution and traditional health 
care, we use the World Wide Web 
as a springboard to offer a broad 
spectrum of products and services 
to the US and abroad. We 
welcome your suggestions as we 



grow to serve you better (http:// 
www.mededge.com/whomiss.htm). 

The mission is readily apparent. It is 
obviously a new service, even by Internet 
standards. 

Parameters of Service 

The service offers e-mail answers to 
medical questions within 48-72 hours. 

The service answers 

any type of medical question you 
might have - whether it is a question 
about a current medical problem, a 
hypothetical question about a 
medical issue that concerns you or 
even general questions you might 
have from within the world of 
medicine (http://www.mededge. 
com/questarch.htm). 

Its advertised answers range from the 
simple to the complex. AnswerDoc says: 

Answer Doc allows you to pose 
questions to medical professionals 
just as you would ask your own 
medical doctor. This offers answers 
to your personal health questions in 
a format that meets your needs and 
pocketbook. Our health care 
professionals are trained to answer 
your simple or complex medical 
questions - not designed to be a 



1 Additional information was solicited from The Medical Edge but no response had been received by 
June 9, 1999. 
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substitute for a physician visit, but a 
valuable resource for medical 
information that can save you time, 
trouble and worry when you need it. 
Services are available for 
immediate phone response, or 48-72 
hour e-mail. In a situation where 
your doctor is unavailable or even 
unable to answer your medical 
questions we try to bridge the gap 
and provide you with the 
information you need (http://wvm. 
mededge.com/answer.htm). 

The AnswerDoc home page has links to 
sample questions, archives, rates, and the 
query form. 

AnswerDoc allows three levels of 
questions. The first level costs $39. It 
allows a telephone response immediately. 
The question may be simple or complex. 

The second level costs $29 and guarantees 
an e-mail response within 72 hours. The 
third level is free. There is no assured 
response time, there is no guarantee of an 
answer, and the user agrees that her 
question and response may be placed in the 
public archive anonymously. The free 
service imposes a 100-word limit to the 
question and answer. 

Identifying the Clients 

The intended user is anyone who would 
use any other medical service, who 
possesses sufficient computer access and 
skills, and who, for using the fee-based 
service, has a credit card. 

Query Form . 

There does not seem to be a standard 
query form for the free service. The client 
is first given the option of the three levels of 
service. In response to his choice, he 
receives a disclaimer page. In substance, 
the lengthy disclaimer denies that the 
service is "practicing medicine," states that 
the information provided is purely 
informational, insists that it is in no way 



attempting to supplant one's personal 
medical care and cannot be considered as 
such, and generally implies that the user is 
not to rely on a word the service provides 
and that in any case the service is not liable 
for anything resulting from such reliance. 

For the fee services, the client must provide 
the usual credit information first. For the 
free service, after he agrees to the 
disclaimer, the move is to a blank screen. 

As noted, the user may access the fee 
service by telephone or e-mail, the free 
service only by e-mail. 

Responsibility to Clients 

In addition to the services and 
disclaimers noted above, the services 
promises response by "medical 
professionals," the "staff," "board certified 
physicians," although it provides no 
information about actual personnel other 
than brief biographies of the chief officers 
of the corporation. No fee request was 
submitted for this review, so it is not known 
if a medical professional is identified in the 
response or in the question negotiation. It 
seems obvious that a telephone interview 
would yield a real person. The free question 
submitted for the review was not answered. 
The user is assured of privacy and 
confidentiality. 

Ease of Use 

AnswerDoc is highly visible on The 
Medical Edge's home page with a link to a 
page describing AnswerDoc itself. A link to 
the query form would be five links from the 
Medical Edge's home page. The Medical 
Edge's home page identifies all avenues of 
service, information, and products, and for 
that reason is rather crowded. Although I 
am very skeptical of the purported altruism 
of "Mededge," (so far the free information 
request leads to a blank page, and the 
"archive" exists only in theory), the home 
page leads to a wealth of information, 
including a site medical reference file with 
18 overall entries (e.g. mental health, 
women's health, cancer), each with 20 or 
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more entries), a hot link to Medline, a 
medical dictionary, and a listing of support 
networks. One of the services is 
"SelectDoc," in which one enters a profile 
for a physician and then receives a list of 
physicians approximating the profile. 

Administration 

Integrated Medical Technologies, Inc., 
the firm operating The Medical Edge, is a 
for-profit entity, so funding sources and 
staff input are not a public issue. Likewise, 
the internal workings of the organization are 
not discussed. The Medical Edge 
characterizes itself as friendly and invites 
public comment. No response has been 
received from the service about the blank 
page retrieved by requesting the free 
service. 

There are not ethical problems per se, 
but there may be considerable question 
concerning liability of the entity for wrong, 
incomplete, or even harmful information. 
The service claims not to "practice 
medicine," but it certainly presents itself as 
qualified to give reliable medical opinions. 

On the other hand, the organization 
makes clear that it does not "diagnose," and 
that it is not an emergency room, and that it 
will make arrangements with an emergency 
room or a 91 1 if that seems appropriate. 

The company claims copyright 
(statutory and common law) on its own 
online material and name. 

Stafflng and Training 

There are no volunteers. All staff who 
respond to questions are medical 
professionals qualified to give advice on 
medical matters. The site provides no 
information about any training in 
responding to questions. 

Answering Questions 

No information is given about the 
internal process of assigning questions and 



answering questions. Medical questions are 
not referred as such. Users are referred 
locally if the company feels that there is a 
medical emergency. 

Inventory Control 

No information is given about inventory 
control at the site. 

Coping with Demand 

One of the company's selling points is 
the availability of the service when a regular 
physician is unavailable. The sample 
questions talk about the regular doctor's 
being “away for the weekend." But a note at 
the beginning of the $39 track indicates: 

Please Note: This service will not 
be available from 6 p.m. Friday 
evening until 12 midnight Saturday 
Night. The service will otherwise 
be available 24 hours a day. We 
regret any inconvenience this may 
cause. 

This note dilutes the "ever present" tenor of 
the overall message. Perhaps there is a 
religious element here, or perhaps this 
particular period is one of high emergency 
which this service does not deal with 
anyway. Whatever the reason, it does seem 
to depart from the service's mission 
statement. 

There is no discussion of an overload 
scenario, nor any of a maximum number of 
questions the company can handle. 

Archiving 

This seems to be a very new, 
well-intentioned company which fully 
expects to have a large and sought-after 
archive. At this point, however, AnswerDoc 
invites disappointment because there is no 
archive. It would have been better to make 
no mention of archiving or to discuss it only 
in general terms, rather than to set it up as a 
choice that does not exist. 




64 



70 



There is no discussion, as yet, as to how 
the archive, when it is in place, will be 
indexed and referenced. 

Quality Control 

This would seem to be a most important 
aspect of this for profit organization, but 
there is no information about how quality 
control is maintained at the web site. 

As noted, there is no public aspect to 
quality control. The best mark of quality 
will be if there is a steady increase in repeat 
users, which presumably would enhance the 
bottom line. The quality of the staff is a 
major aspect of quality control, but there 
also is no publicly accessible information 
about how the company selects, or deals 
with (whether as employees, consultants, 
contract or commission) medical 
professionals, and the qualities it looks for 
in its staff. 

Hardware/Software 

There does not appear to be a text-only 
version of the web site. Java script does not 
appear to be necessary. There is no 
information about hardware/software 
support at the web site. 

Acknowledgment 

The acknowledgment to a question 
appears to be the answer itself so that it is 
either immediate (unless Friday at 6 until 
Saturday midnight) or within 72 hours. 
There is no indication at the web site of any 
acknowledgment for free questions. 

Question Negotiation 

There is question negotiation, 
depending on the information required by 
the service in order to answer questions 
satisfactorily. There is no stated limit as to 
time. This may not be true for the free 
question service, since the emphasis seems 
to be on curtailing the length of question 
and response of free questions. 



Response Guidelines 

The response is primarily dictated by 
which track the user picks. Some questions 
still remain about responses to fee-based 
services: Can the user call back with more 
information, or will that be another $39? In 
other words, when is the question 
answered? The same is true for the $29 e- 
mail: Does the user have the opportunity to 
clarify the question at no extra charge? 
When the medical professional fielding the 
question sees that the question cannot be 
answered with the information available, or 
that the user is clearly “barking up the 
wrong tree”, as it were, will the user be 
expected to face another $29 fee for 
correcting the original mistake? 

This is a natural language service 
oriented toward individual and original 
responses to real questions about health. 
From the sample answers, it seems that 
documentation (at least any extensive 
documentation) will probably not be 
offered, although, if physicians make use of 
the service, as envisioned, there will 
probably be more attention given to sources. 
Thus the type or category of user is a factor 
may become a factor in documentation 
guidelines. Policy limitations on sources 
used by the service will probably follow the 
copyright guidelines used by the service 
itself. Hopefully, archives will become a 
reality so that previous answers can be re- 
used. Responses are by e-mail and 
telephone. 

Evaluation 

There is no mention of evaluation done 
by the company or of the statistics they 
maintain and whether they are publicly 
available. 

External Recognition 

The site does not announce that it has 
won any awards. 

Comment 

I think that the whole concept of online 
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medical assistance as enunciated in The 
Medical Edge’s mission statement will rest 
in part on how seriously the company takes 
its free service. If it is just bait, then it will 
descend to an online gimmick. If the 
company energizes its free service and 
develops its archive as a public resource, 
then its image will be considerably 
enhanced and the efficacy of the endeavor 
will have been established. The company 
has an admirable vision, and, if it can do 
reasonably well and maintain and enhance 
its public service component, everyone will 
benefit. 



ScienceNet: Evaluative Analysis 



by Jennifer R. Heiland' 



Mission, Objectives, Statement of 
Purpose 

The expressive name of this United 
Kingdom-based service is ScienceNet. It is 
a free question and answer service on the 
Internet, located at http://www.sciencenet. 
org.uk. According to the service's 
homepage, "ScienceNet is a free science 
information service staffed by scientists 
who are experts in explaining complex 
topics in everyday language." Since this 
purpose statement is located directly 
underneath the title "ScienceNet" on the 
homepage, it is apparent to the user what 
this service is about. It also refers to itself 
as "Your One-Stop Science Site" in a 
banner across the top of its homepage, 
making it more obvious that this service 
concentrates on science. 

Parameters of Service 

No limitations are made on types of 
questions answered. The scrollable box for 
a question on the query form is quite large 
(approximately six inches by three inches), 
which encourages lengthy questions. Also, 
there is a statement, "please try to make the 
question as specific as possible" that would 
indicate the service answers lengthy 
questions. On the homepage, the subject 
categories of questions are given: 
Archaeology & Palaentology, Biological & 
Medical Sciences, Chemistry, 



Environmental Science, Geography & Earth 
Sciences, Mathematics & Computing, 
Physics & Computing, Physics & 
Astronomy, Sociology & Psychology, 
Technology & Engineering which would 
give prospective users an idea of what 
topics this service covers. The inclusion of 
Archaeology, Sociology, and Psychology is 
interesting in that these topic areas are not 
typically considered to be part of "science." 
The user is asked to search the database 
before submitting a question. 

Identifying the Clients 

It is stated on the query form page that 
the service at times has a large workload 
and that, since their funding comes from 
U.K.-based organizations and companies, 
U.K. users are given priority. The statement 
is made that, when the workload is less, the 
service tries to answer questions from 
everyone. It is not clear to the user whether 
questions submitted during busy times 
merely get shunted to the back of the queue 
or if they are rejected completely. Although 
U.K. users are given priority, everyone is 
encouraged to use this service. 

Query Form 

To submit a question, the user is 
requested to fill in all the fields given on the 
query form, although only the return e-mail 
address is labeled "compulsory." These 



^ As of June 10, the author was unable to speak with anyone at ScienceNet and had received no 
response to e-mail questions about the service. Some of the conclusions and inferences in this paper may 
be inaccurate due to missing data. 
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fields include: 



1 . Given Name, Family name. 

2. Question . 

3. Sex. 

4. Age. 

5. Phone number. 

6. Address. 

7. Country. 

8. Whether the person is studying science 
and, if so, at what level. 

9. Whether the question is related to 
studies. 

10. Whether the user has used services 
before via e-mail or phone . 

1 1 . Where the user heard of the service. 

12. Reply e-mail address. 

Unfortunately, no provision is made for 
questioners unable to use forms (i.e., no 
separate e-mail address for queries is 
provided), although the phone number for 
ScienceNet's sister service. Science Line, 
which answers similar queries via the 
telephone is give both for U.K. users and the 
"rest of the world" with hours the hotline is 
open specified. Questioners are encouraged 
to "please try to make the question as 
specific as possible," the only guidance 
given on how to ask the question. It is also 
possible to submit a question to ScienceNet 
via a query form on the BBC web site. The 
two forms are essentially the same except 
that on the ScienceNet form it asks whether 
the question is related to one's studies and 
on the BBC form the questioner has the 
option to check three boxes agreeing to 
one's question, name and/or e-mail address 
being published. Since none of these except 
the question is included in the ScienceNet 
archive it is unclear why these options are 
needed. 

Responsibility to the Clients 

Little detail is provided directly to the 
user about what will happen to his question. 
It is not evident from the query form page 



what will happen to the question after it is 
submitted. No mention of a time frame is 
given anywhere. On a separate page, which 
requires some scrolling and two clicks from 
the homepage, one finds that a 
multi-disciplinary team of scientists answers 
questions as well as referring questions to 
over 2000 scientists worldwide. The 
question-answering process is not 
addressed, although user privacy is. 
According to the query form page, 
"ScienceNet and Science Line work under a 
strict code of privacy so none of your details 
will be sold or passed on to any other 
organization or company and they will be 
deleted after your question is answered." 

No medical/legal disclaimer is made on any 
of the web pages. 

Ease of Use 

The link to the query form page is on 
the lower right comer area of the homepage 
as an underlined link entitled, "Ask A 
Question." It is easy to find and once one 
gets to the query form page, the service 
states, "this page allows you to send a 
question to the scientists at ScienceNet." It 
does ask the user to search the database of 
previously answered questions prior to 
submitting a question. It also gives the 
telephone number of ScienceNet's sister 
service. Science Line. There is no other 
option for submitting a question other than 
the e-mail form (unless one uses the Science 
Line service, a related, but separate service). 
The e-mail address of the query form is not 
evident to the user. Thus, unfortunately, 
there are no plans made for users with web 
browsers unable to use forms. 

Administration 

Various sponsors finance ScienceNet. 
The primary sponsor is the United 
Kingdom’s National Lottery's Charities 
Board, followed by the Wellcome Trust, the 
“largest non-govemmental funder of 
medical research in the United Kingdom,” 
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according to ScienceNet. There are four 
lesser sponsors, all of whom are based in the 
United Kingdom as well as nine other 
British partners from whom ScienceNet 
receives non-fmancial support. The service 
also receives small amounts of support from 
twelve different groups such as the BBC 
and IBM. Since there are so many different 
groups supporting them, it is doubtful that 
any one has much influence as to affect 
objectivity. Many of these various 
supporters and partners are recognized 
worldwide for their quality and likely have 
only positive effects on the quality of 
ScienceNet. Through the support of its 
many sponsors, ScienceNet and its 
sister-service. Science Line, are free 
services. 

Staffing and Training 

There are ten staff members, seven of 
whom would seem to be involved in 
actually answering questions (on the staff 
page or on their own personal pages they 
indicate a subject specialty), although they 
may all answer questions (it is unclear). 
Questions are answered by these permanent 
staff members if they are able (staff 
members range in level of science education 
from an undergraduate degree to a Ph.D.) 
and are referred to over 2000 scientists 
working in the U.K. and around the world if 
the permanent staff is unable to answer the 
question. 

The staff has one manager who 
manages both Science Line and ScienceNet. 
The service is free. The same people who 
are answering e-mail questions with 
ScienceNet staff the phone for Science Line, 
a free telephone service which "offers 
members of the public" opportunity to 
obtain authoritative answers to their 
scientific queries (http://www.sciencenet. 
org.uk/intro/aboutscinet.html). 



Some of the volunteer experts for the 
service other than permanent staff come 
from organizations such as the Science 
Museum Library, the Natural History 
Museum, and the Royal Botanic Gardens, 
Kew, all located in the United Kingdom. 
People interested in being a volunteer to 
answer questions with ScienceNet and 
Science Line are encouraged to apply. An 
online form is provided which states that 
one need not be a "Professor" but should 
have a comprehensive knowledge of an area 
of science, technology, engineering or 
medicine (http://www.sciencenet.org.uk/ 
intro/forms/expert.html), which likely 
means at least an undergraduate degree in a 
science or related field. The staff answerers 
have ScienceNet and Science Line as their 
main responsibilities at this organization. A 
general impression from reading the various 
personal homepages is that many of the 
answerers have other time commitments 
such as being a graduate student or having 
other jobs, so being an answerer for 
ScienceNet and Science Line is evidently 
only a part-time job. 

No mention is made of an expert system 
being used for question allocation and since 
most of the questions are answered in-house 
(approximately 70 percent), the staff are the 
ones who must decide when they require 
more in-depth expert assistance. From the 
phrasing on the web-site and other sources, 
it is implied that, even when they consult an 
expert, ScienceNet still serves as an 
intermediary between questioner and the 
volunteer answerer. 

The general qualifications specified for 
the volunteers (comprehensive knowledge 
of an area of science, technology, 
engineering, or medicine) probably hold 
true for the regular staff, although since 
ScienceNet and Science Line are committed 
to "explaining complex topics in everyday 
language" then it is likely that this ability is 
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a prerequisite to being a regular staff 
member answering questions. Related to 
this, at least two of the staff members have a 
Masters of Science in Science 
Communication, presumably a degree that 
would cover aspects of making science 
intelligible to the specialist and 
non-specialist. 

No discussion is made of time 
commitment, training, or seniority on the 
web site or in other literature consulted. 

No ethical statements are given. 

Privacy and liability statements are 
discussed above. There are also no 
statements warning of possible out-of-date 
information, especially in medical areas or 
warning statements/caveats to contact one’s 
doctor, lawyer, etc. 

Answering Questions 

The apparent process a question goes 
through from receipt to response is 
described below. First, a person sends an e- 
mail query to the ScienceNet via the query 
form page (http://www.sciencenet.org. 
uk/intro/forms/askaq.html). ScienceNet 
receives it, and it receives priority if the 
questioner is from the United Kingdom. It 
is specified in other areas, including the 
Virtual Reference Desk entry for this 
service, that ScienceNet tries to answer 100 
percent of questions asked even if the 
answer is already present in the database, so 
presumably, although foreign queries are at 
the bottom of the list, they will be answered 
eventually. 

It is not clear whether individual staff 
members answer questions in all areas or 
only in specific areas of science. However, 
from the information available concerning 
Science Line, it is known that staff answer 
questions outside their area of specialty for 
that service, so they probably do so as well 
for ScienceNet. ScienceNet states that 



questions are answered by a multi- 
disciplinary team of scientists who staff the 
service, so it may even be that answers are 
discussed and answered in a group, 
especially for confusing questions. 

If no one on the ScienceNet service is 
able to answer the query, then they refer the 
question to an outside scientist volunteer. It 
seems that the responses then would be 
filtered through the staff at ScienceNet to 
make sure it is clear and understandable for 
the questioner. This review could be a form 
of quality control. The answer is then sent 
to the questioner. 

Since phone number is asked for in the 
e-mail query form, it may be an option for 
question negotiation (although perhaps not 
for foreign questioners), but this is not 
explicitly stated anywhere. No response 
guidelines are included at the site regarding 
personalization, personal contacts, time to 
spend on individual questions, response 
time, etc. Since the service tries to answer 
all questions, even ones already existing in 
the database, canned responses turning away 
persons due to not matching client base or 
types of questions answered probably do not 
exist. However, since the staff get some 
repeat questions, they make use of their 
database of previously answered questions 
to answer these repeat questions. If 
information is obviously out of date, they no 
doubt modify the data. 

From the responses in the database and 
articles giving brief statements from 
satisfied users, it seems that the service 
gives both instructional and informative 
answers. From the database it appears that 
they concentrate more on the information/ 
answer itself rather than on instructions for 
finding it. None of the answers sampled in 
the archive included citations to sources. 
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Since the service's goal is to answer all 
questions, presumably all communications 
are eventually acknowledged. ScienceNet, 
which began in December 1994, answers 
approximately 400 e-mail questions per 
week and Science Line has answered 
153,000 calls total since its inception in 
March 1994. It is not specified anywhere 
how long a user should expect to wait for an 
answer to one's question. Multiple 
questions from the same person are 
permitted. 

Inventory Control 

No information is available at the site 
about inventory control. 

Coping with Unexpected Demand 

No information is known about the 
policy or procedures for coping with 
unexpected demand except that when the 
service is especially busy, non-U.K. 
questions will be sent to the back of the 
queue. 

No general emergency procedures are 
evident. However, while I was researching 
this paper, the search engine for their 
database of past questions was down, which 
was indicated in bright red all capital letters 
at the web site. Unfortunately no dates were 
given to indicate how long it would be down 
and/or had been down. 

Archiving 

In archiving past questions, both 
questions from ScienceNet and Science 
Line are included. The archive is publicly 
accessible and seems to include questions 
since the beginning of Science Line. In the 
archive all that is included is the question, 
question number, and the answer. On the 
ScienceNet web site, there is no option for 
the questioner to not have his question 
included in the archive, but since no 
identifying information is included in the 
archive, including a question is presumably 



not a problem for an individual questioner. 
The archive of past questions serves both to 
inform the general public and to serve as an 
aid for staff who can refer to it for 
answering questions. It is easily accessible 
from various pages, most easily from the 
query form page. The form asks a user to 
search the database prior to submitting a 
question. 

The database is usually accessible in 
two ways, a classified browsable 
arrangement and a search engine. With the 
search engine down as it is at present, the 
only way to access the archive is by 
browsing the classified arrangement. When 
the search engine is available again, one can 
search the entire database, or one of nine 
science subdivisions. The user can specify 
the number of results, the order (relevance 
or by date), the level of detail in the 
description, and the keywords to search on. 

The need to use only British spellings 
and the lack of ability to use Boolean logic 
to link search keywords are both negative 
aspects to this search engine. After an 
answered question has been put in the 
database, the answer will eventually be re- 
written at three separate levels of difficulty 
(primary school, secondary school, and 
post-secondary school). Two good 
examples of re-written answers are the 
answers to "Do other planets other than 
Earth have volcanoes?" (#p00885a) and 
"What is an Eclipse?" (#p00921a). These 
can be viewed by following the path 
Astronomy and Physics/Earth/ from the 
ScienceNet homepage. If no level is 
specified for the question, then it is at the 
level of difficulty matching the original 
questioner. Most answers remain in their 
original state at the present time. It appears 
that the staff at ScienceNet manage the 
archive. 
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Quality Control 

No information about quality control is 
available at the web site. 

Hardware/Software 

No information about 
hardware/software is at the web site. 

Acknowledgment 

There is no indication of 
acknowledgment to the user on receipt of a 
question, and it was not possible to verity 
this by submitting a question. 

Question Negotiation 

There is no evidence of question 
negotiation. 

Response Guidelines 

From the answers in the archive, it 
appears that the responses to questions are 
in the form of free writing with no standard 
format. Answers varied in length and the 
amount of personal judgment, although 
some of the questions called for judgment 
(e.g., whether a vegetarian diet is better or 
worse for you than a diet that includes meat) 
which the service provides. The service can 
provide both informational and instructional 
answers but tends toward informational. 

This proclivity may be a response to the sort 
of questions asked. It would seem that, if a 
questioner specified wanting instructional 
information, she would receive it. 

No documentation was cited on any of 
the questions examined, and although this 
was a small subset of the questions, it did go 
across various disciplines and sub- 
disciplines. The level of knowledge of the 
user is presumably ascertained by the age of 
the questioner and tone of the question. 

One of ScienceNet's major goals is to make 
complex topics understandable in everyday 
language, so it tries to match the level of the 
answer to that of the questioner. Answers 
from prior questions are recycled when 



possible. No further information regarding 
response guidelines is available. 

Evaluation 

No data regarding evaluation are 
available. 

External Recognition 

ScienceNet refers to itself and its sister 
service Science Line as "award winning" 
but specific awards are not listed at the site. 

Comment 

Overall, I found this service good. 
Although it has some faults, such as only 
being able to submit questions via a form 
(no e-mail address access for queries), no 
specified time frame for answering queries, 
and no citation to sources, it does meet its 
major goal specified on its home page, to 
explain "complex topics in everyday 
language." 
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Appendix A. Framework for Analysis of an 
Electronic Question/ Answer Service 

Version 1.0 (June 4, 1999) 



INPUT; Communications from Client a question? (e.g., examples of 

questions, complete sentences only) 



1. Mission, Objectives, Statement of 
Purpose 

1 . 1 What is the name of this service? Is 
the name expressive? 

1 .2 What is the URL? (For assessment) 

1 .3 What is the mission/purpose of this 
service? 

1 .4 Is the mission/purpose apparent to the 
user? 

2. Parameters of Service (what kinds 
of questions does the service 
accept/answer?) 

2. 1 What types of questions does this 
service answer? (ex: short answer, 
long answer, unique) 

2.2 Does the service reject questions? If 
so, what is the basis for rejection? 
(e.g. outside of scope, time limited) 

2.3 What limits exist on the service for an 
individual user? Consider: 

- number of questions per message 

- number of questions per user 

3. Identifying the Clients 

3.1 Who are the intended users? What 
are their characteristics? (e.g., age, 
gender, affiliation, location) 

4. Query Form 

4. 1 What information does is required to 
answer a question/What must the user 
provide?.(name, e-mail address, 
phone number, question, sources 
already used, date needed by, grade) 

4.2 What options are available for 
sending a question? (e-mail, query 
form, both) 

4.3 What guidance is given on how to ask 



5. Responsibility to the Clients 

5. 1 What does the service indicate it will 
provide to the client? 

5.2 Does it inform the client about: 

- how soon an answer will be 
returned? 

- who answers the question? (i.e. 
volunteers, professionals, experts, 
other users?) 

- the types/or specific sources it uses? 
For any of these, where and how? 

5.3 Does it explain the question- 
answering process? Where and how? 

5.4 Does the service address user 
privacy? How? 

5.5 Is there a statement of liability/ 
responsibility/appropriate use? (E.g. 
medical, legal info; accuracy) 

5.6 What is included in it? 

6. Ease of Use/Instructions to User 

6. 1 How is the service positioned on the 
web page in relation to the other 
services on the site? 

6.2 Is it easy or hard to find? (Consider 
logic of links, name of service.) 

6.3 Does it direct you to other sources 
(i.e. FAQs, search the archives, links 
to other services) 

6.4 Is the destination address apparent to 
the user of a form query? 

■6.5 What options (e.g. e-mail, fax, phone) 
are available for delivery of 
information? Who selects the mode 
used and on what basis? 



THRUPUT 

7. Administration 

7. 1 How is the service ftinded? 

7.2 How does fiinding/sponsorship 
influence quality and/or objectivity? 

7.3 Is service fee-based? If so, what is 
the fee structure and how are the fees 
collected? 

7.4 How is staff organized? 

7.5 Who allocates questions to 
answerers? Is it an expert system or 
human? 

7.6 Are there ethical (e.g., dan- 
gerous/criminal questions) state- 
ments? Are they easily publicly 
accessible? 

7.7 Are there privacy statements? Are 
they easily publicly accessible? 

7.8 How are copyright, fair-use, selective 
dissemination of information issues 
managed? 

7.9 Are there liability disclaimers and/or 
content disclaimers? 

7. 1 0 Are there general emergency 
procedures (e.g., for system failure)? 

7.11 To what level are staff involved in 
policy development? 

8. Staffing and Training 

8.1 Who will answer questions? 
(Volunteers/staff) 

8.2 What level of expertise is required? 

8.3 How are answerers recruited and 
selected? 

8.4 What is expected/average time 
commitment for answerers? 

8.5 What training exists for the 
answerers? 

8.6 Are there seniority benefits and/or 
increased responsibilities? 

8.7 If answers are staff, how does the Q/A 
service responsibilities fit into their 
other responsibilities? 

9. Answering Questions 

9.1 Describe the process a question goes 
through from receipt to response. 

9.2 Are there question negotiation 



guidelines? Is communication 
through other media (e.g., phone) an 
option for negotiation? 

9.3 What templates or canned answers 
exist, if any? 

9.4 How much personalization is given in 
responses? 

9.5 Are personal contacts to answerers 
given or encouraged? 

9.6 What guidelines exist about the nature 
of the response? Consider: 

- informational/instructional 

- level of difficulty 

- degree of detail 

9.7 What guidelines exist for response 
time, if any? 

9.8 What is the policy and procedure for 
referring a question, if any? To 
whom are referrals made? 

9.9 Are sources of answers given? 

10. Inventory Control 

10.1 How do questions accumulate? Are 
questions assigned? selected by 
answerers? queue? 

10.2 How are questions prioritized? What 
criteria are used? 

10.3 What guidelines are there for response 
time for answering questions? 

10.4 What is done with questions or clients 
not matching guidelines? 

10.5 Are all communications 
acknowledged? How quickly? 

10.6 What are the guidelines on time limit 
to be spent on answering individual 
question, if any? 

11. Coping with Unexpected Demand 

11.1 Are there scheduled down-times? 

1 1 .2 How are down-times and other 
interruptions of service com- 
municated to users? 

1 1 .3 What is done if system is inundated 
by demand? Are there backup staff, is 
it closed down, delayed response 
time, etc.? 

1 1 .4 Is there a limit to the number of 
questions the system can handle? 
What is that limit? 
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12. Archiving 

12.1 What is the nature archive, if there is 
one? Consider: 

- private/publicly accessible. If 
private, accessible to whom? 

- kinds of information included 

- length of time materials are archived 

- purpose of archive 

- degree of anonymity maintained 

12.2 Are users allowed to specify if they 
do not want their question archived? 

12.3 How easily accessible is archive if it 
is public? 

12.4 Is archived indexed and/or 
searchable? If so, how? 

12.5 Is archive selective? By date, or con- 
tent? 

12.6 Is there a FAQ? Is it indexed or 
searchable? 

12.7 How is FAQ related to archive? 

12.8 Who manages the archive? 

13. Quality Control 

13.1 How is quality control maintained, if 
there are procedures? Consider, for 
example, using standard resources for 
answering questions (written, people, 
electronic)? 

1 3 .2 What happens if quality is considered 
inadequate? 

13.3 Is there a review process for new 
answerers, for answers? 

14. Hardware/software 

14.1 Describe the hardware/software used 
by this system. 

OUTPUT: Communications to Client 

15. Acknowledgment 

15.1 What kind of acknowledgment is sent 
when a query is received, if any? 

1 5.2 How soon is it sent? 

1 5.3 What information does it contain? For 
example, 

- restatement of the question 

- expected response time 



15.4 Is the acknowledgment generated 
automatically or with human input? 

16. Question Negotiation 

1 6. 1 If the question requires clarification, 
what are the guidelines for 
clarification? Consider: 

- the number of messages needed to 
clarify the question 

- modes to be used for 
communicating 

- any limitations on time 

17. Response Guidelines 

1 7.1 How is the response generated: form, 
free writing, or combination of both? 

1 7.2 Does the service use standard 
language to respond in full or part to 
questions? If so, solicit examples. 
This allows you to see the kinds of 
questions for which this is available. 

1 7.3 What are the limitations or constraints 
on the answers? Consider issues of 

- brevity 

- fact/judgment 

- for bibliographic citations: number 
and format 

- instruction, actual information or 
both. If the latter, what factors 
influence the judgment about which 
to provide? 

1 7.4 What documentation accompanies the 
answer, if any? 

1 7.5 How does the service respond to: 

- ineligible questioners 

- inappropriate questions 

1 7.6 How does the service take into 
consideration factors that may 
influence choice of answer, e.g. level 
of knowledge or language skills of 
user? 

1 7.7 What policy restrictions are there on 
the sources which can be used to 
answer the questions, if any? 

17.8 Are previous answers available and 
searchable so that answers can be 
reused or adapted for other questions? 
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1 7.9 What format does the response take: 
direct e-mail or bulletin board 
posting? 

17. 10 How does the questioner request 
clarification of the answer? 

1 7.1 1 How does the client request follow- 
up? 

17.12 Is there a policy on response time? 
What is considered the range of 
acceptable response times? 

17.13 Is there a protocol for determining 
priority in processing answers? If so, 
what is it? 

17.14 What are the policies regarding 
attaching or referring to non-textual 
materials? 

18. Evaluation 

18.1 Is there a formal evaluation procedure 
in place? 

1 8.2 What is the nature of that procedure? 

18.3 How frequently does it occur? 

1 8.4 What measures of performance are 
used? Consider, for example: 

- user satisfaction 

- accuracy 

- time lag 

- response rate (percentage of 
questions answered) 

1 8.5 Is there follow-up with the user? If 
so, in every case or in selected cases? 

1 8.6 How does the service evaluate the 
quality of the answers provided? 

18.7 Does the service meet mission goals? 

18.8 What statistics are maintained by the 
service regularly? How are they 
used? 

19. External recognition 

19.1 What awards has this site received? 

19.2 What articles are written about it? 
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